-- Logs begin at Fri 2022-09-09 19:48:45 UTC, end at Fri 2022-09-09 20:47:50 UTC. -- Sep 09 20:13:18.028087 dv-cinder-849325-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 09 20:13:18.038416 dv-cinder-849325-4-iscsi devstack@keystone.service[81418]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 09 20:13:18.040434 dv-cinder-849325-4-iscsi devstack@keystone.service[81418]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Sep 9 20:13:18 2022] *** Sep 09 20:13:18.040434 dv-cinder-849325-4-iscsi devstack@keystone.service[81418]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 09 20:13:18.040434 dv-cinder-849325-4-iscsi devstack@keystone.service[81418]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 09 20:13:18.040434 dv-cinder-849325-4-iscsi devstack@keystone.service[81418]: nodename: dv-cinder-849325-4-iscsi Sep 09 20:13:18.040434 dv-cinder-849325-4-iscsi devstack@keystone.service[81418]: machine: x86_64 Sep 09 20:13:18.040434 dv-cinder-849325-4-iscsi devstack@keystone.service[81418]: clock source: unix Sep 09 20:13:18.040434 dv-cinder-849325-4-iscsi devstack@keystone.service[81418]: pcre jit disabled Sep 09 20:13:18.040434 dv-cinder-849325-4-iscsi devstack@keystone.service[81418]: detected number of CPU cores: 4 Sep 09 20:13:18.040434 dv-cinder-849325-4-iscsi devstack@keystone.service[81418]: current working directory: / Sep 09 20:13:18.040434 dv-cinder-849325-4-iscsi devstack@keystone.service[81418]: detected binary path: /usr/bin/uwsgi-core Sep 09 20:13:18.040981 dv-cinder-849325-4-iscsi devstack@keystone.service[81418]: your processes number limit is 47680 Sep 09 20:13:18.040981 dv-cinder-849325-4-iscsi devstack@keystone.service[81418]: your memory page size is 4096 bytes Sep 09 20:13:18.040981 dv-cinder-849325-4-iscsi devstack@keystone.service[81418]: detected max file descriptor number: 2048 Sep 09 20:13:18.040981 dv-cinder-849325-4-iscsi devstack@keystone.service[81418]: lock engine: pthread robust mutexes Sep 09 20:13:18.040981 dv-cinder-849325-4-iscsi devstack@keystone.service[81418]: thunder lock: enabled Sep 09 20:13:18.040981 dv-cinder-849325-4-iscsi devstack@keystone.service[81418]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 09 20:13:18.040981 dv-cinder-849325-4-iscsi devstack@keystone.service[81418]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 09 20:13:18.079735 dv-cinder-849325-4-iscsi devstack@keystone.service[81418]: Python main interpreter initialized at 0x55588ebef9c0 Sep 09 20:13:18.079735 dv-cinder-849325-4-iscsi devstack@keystone.service[81418]: python threads support enabled Sep 09 20:13:18.079735 dv-cinder-849325-4-iscsi devstack@keystone.service[81418]: your server socket listen backlog is limited to 100 connections Sep 09 20:13:18.079735 dv-cinder-849325-4-iscsi devstack@keystone.service[81418]: your mercy for graceful operations on workers is 90 seconds Sep 09 20:13:18.080046 dv-cinder-849325-4-iscsi devstack@keystone.service[81418]: mapped 403077 bytes (393 KB) for 2 cores Sep 09 20:13:18.080135 dv-cinder-849325-4-iscsi devstack@keystone.service[81418]: *** Operational MODE: preforking *** Sep 09 20:13:18.080400 dv-cinder-849325-4-iscsi devstack@keystone.service[81418]: *** uWSGI is running in multiple interpreter mode *** Sep 09 20:13:18.080400 dv-cinder-849325-4-iscsi devstack@keystone.service[81418]: spawned uWSGI master process (pid: 81418) Sep 09 20:13:18.080579 dv-cinder-849325-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 09 20:13:18.081263 dv-cinder-849325-4-iscsi devstack@keystone.service[81418]: spawned uWSGI worker 1 (pid: 81429, cores: 1) Sep 09 20:13:18.081485 dv-cinder-849325-4-iscsi devstack@keystone.service[81418]: spawned uWSGI worker 2 (pid: 81430, cores: 1) Sep 09 20:13:18.081623 dv-cinder-849325-4-iscsi devstack@keystone.service[81418]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 09 20:13:22.326608 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 20:13:22.326608 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: from cryptography.utils import int_from_bytes Sep 09 20:13:22.327427 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 20:13:22.327427 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: from cryptography.utils import int_from_bytes Sep 09 20:13:22.327551 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 20:13:22.327551 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: from cryptography.utils import int_from_bytes Sep 09 20:13:22.327968 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 20:13:22.327968 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: from cryptography.utils import int_from_bytes Sep 09 20:13:22.379025 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 09 20:13:22.379282 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 09 20:13:22.379378 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] command line args: [] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 09 20:13:22.379888 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 09 20:13:22.380074 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ================================================================================ {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 09 20:13:22.380332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] admin_token = **** {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.380577 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] config_dir = [] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.380762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.380940 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] config_source = [] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.381118 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] debug = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.381344 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.382508 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.382508 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.382508 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.382508 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.382768 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] list_limit = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.382768 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] log_config_append = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.382879 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.384432 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] log_dir = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.384572 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 09 20:13:22.384672 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] log_file = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.384756 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 09 20:13:22.384833 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.384900 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] command line args: [] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 09 20:13:22.384968 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.385028 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 09 20:13:22.385095 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.385159 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ================================================================================ {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 09 20:13:22.385406 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.385480 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] admin_token = **** {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.385550 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.385611 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] config_dir = [] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.385675 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.385735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.385798 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.385857 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] config_source = [] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.385857 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] debug = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.385975 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.386035 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.386117 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.386187 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.386252 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.386311 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.386375 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.386433 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.386490 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.386557 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.386622 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.386680 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] list_limit = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.386737 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.386792 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] log_config_append = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.386848 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.386916 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.386916 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] log_dir = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.387037 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] log_file = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.387099 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.387198 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.387313 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] publish_errors = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.387590 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.387682 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.387765 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.387861 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.387935 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.388001 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.388062 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.388119 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.388262 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.388348 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.388483 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] standard_threads = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.388602 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.388713 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.388813 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.388951 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.389036 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.389116 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.389188 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.389309 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] use_journal = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.389435 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.389519 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] use_json = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.389643 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] use_stderr = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.389727 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.389816 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] use_syslog = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.389941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.390048 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.390168 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.390266 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.390445 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.390525 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.390740 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.390941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.391031 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.391132 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] publish_errors = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.391211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.391372 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.391517 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.391599 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.391672 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.391776 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.391884 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.392025 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.392111 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.392266 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.392344 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.392507 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] standard_threads = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.392592 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.392735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.392828 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.392981 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.393060 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.393162 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.393265 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.393367 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] use_journal = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.393507 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.393588 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] use_json = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.393709 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] use_stderr = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.393787 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.393888 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] use_syslog = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.393966 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.394123 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:13:22.394209 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.394382 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.394460 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.394623 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.394714 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.394885 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.394971 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.395126 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.395210 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.395378 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.395457 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.395622 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.395701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.395866 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.395950 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.396097 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.396175 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.396326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.396406 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.396555 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.396639 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.396787 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.396871 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.397021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.397096 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.397280 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.397377 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.397514 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.397619 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.397743 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.397889 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.397974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.398047 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.398213 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.398301 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.398449 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.398528 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.398693 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.398772 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.398920 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.399006 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.399151 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.399241 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.399382 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.399462 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.399610 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.399695 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.399838 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.399915 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.400070 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.400155 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.400297 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.400387 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.400530 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.400608 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.400773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.400851 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.401027 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.401113 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.401287 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.401381 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.401519 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.401608 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.401752 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.401839 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.401984 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.402069 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.402227 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.402306 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.402453 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.402607 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.402682 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.402758 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.402910 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.402988 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.403143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.403233 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.403376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.403456 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.403609 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.403693 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.403835 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.403922 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.404068 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.404144 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.404297 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.404375 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.404530 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.404643 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.404742 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.404838 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.405001 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.405107 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.405233 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.405397 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.405500 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.405578 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.405725 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.405812 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.405928 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.406022 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.406171 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.406258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.406416 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.406499 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.406648 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.406737 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.406890 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.406968 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.407124 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.407209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.407356 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.407480 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.407589 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.407676 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.407826 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.407909 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.408059 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.408140 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.408293 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.408416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.408532 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.408615 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.408764 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.408842 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.408996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.409075 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.409229 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.409336 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.409480 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.409574 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.409713 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.409794 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.409945 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.410025 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.410181 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.410270 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.410436 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.410517 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.410671 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.410756 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.410913 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.410992 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.411149 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.411230 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.411387 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.411465 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.411624 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.411704 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.411858 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.411945 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.412091 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.412171 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.412324 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.412405 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.412551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.412632 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.412778 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.412857 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.413003 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.413090 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.413233 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.413332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.413478 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.413564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.413716 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.413795 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.413948 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.414031 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.414180 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.414259 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.414416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.414504 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.414666 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.414756 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.414907 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.414985 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.415148 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.415226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.415378 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] auth.external = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.415465 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.415606 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.415692 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.415849 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.415927 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.416091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.416177 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.416328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] auth.password = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.416406 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.417102 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.417183 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] auth.token = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.417360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.417459 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.417594 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.417674 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.417844 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.417958 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.418064 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.418146 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.418297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.418397 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.418542 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.418623 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.418781 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.418860 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.419012 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] credential.caching = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.419104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.419256 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.419337 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.419503 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.419583 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.419748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.419835 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.419989 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.420069 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.420228 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] auth.external = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.420313 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.420458 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.420545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.420703 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.420784 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.420951 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.421031 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.421184 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.421336 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] auth.password = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.421441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] auth.token = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.421520 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.421684 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.421766 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.421922 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.422010 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.422171 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.422251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.422405 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.422485 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.422655 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.422747 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.422886 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] federation.caching = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.422964 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.423123 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.423203 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.423359 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] credential.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.423447 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.423599 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.423678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.423826 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.423909 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.424068 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.424156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.424301 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.424381 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.424550 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.424629 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.424785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.424874 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.425033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.425112 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.425283 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.425377 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.425529 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.425619 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.425761 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] identity.caching = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.425841 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.426005 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.426094 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.426248 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.426337 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.426480 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.426560 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.426712 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.426792 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.426954 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.427035 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.427183 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.427269 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] federation.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.427419 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.427499 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.427660 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.427793 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.427921 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.428045 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.428175 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.428301 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.428503 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.428634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.428760 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.428887 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.429024 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.429152 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.429361 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.429490 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.429619 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.429745 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.429870 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.429999 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.430134 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.430264 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.430390 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.430519 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.430643 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.430768 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.430900 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.431030 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.431169 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.431299 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.431427 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.431555 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.431692 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.431822 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.431950 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.432080 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.432839 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.432966 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.433104 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.433235 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.433395 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.433536 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.433666 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.433796 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.433935 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.434065 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.434194 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.434322 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.434450 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.434579 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.434716 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.434851 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.434974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.435104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.435233 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.435380 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.435519 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.435655 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.435787 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.435787 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.436031 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.436159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.436296 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.436426 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.436554 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.436683 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.436812 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.436812 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.437064 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.437203 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.437343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.437470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.437597 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.437733 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.437864 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.437991 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.438122 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.438249 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.438375 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.438508 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.438635 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.438762 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.438904 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.439032 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.439158 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.439293 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.439421 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.439547 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.439672 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.439799 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.439927 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.440062 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.440189 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.440314 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.440439 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.440565 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.440700 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.440834 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.440963 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.441091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.441218 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.441359 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.441488 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.441627 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.441757 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.441886 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.442014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.442014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.442259 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.442396 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.442539 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.442668 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.442797 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.442925 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.443055 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.443192 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.443322 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.443450 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.443579 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.443708 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.443836 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.443973 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.444104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.444244 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.444374 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.444502 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.444631 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.444767 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.444898 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.445026 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.445156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.445330 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.445456 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.445592 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.445726 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.445853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.445979 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.446115 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.446244 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.446379 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.446507 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.446634 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.446761 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.446886 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.447012 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.447146 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.447274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.447401 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.447527 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.447652 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.447788 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.447922 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.448049 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.448174 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.448302 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.448429 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.448429 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.448675 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.448802 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.448937 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.449062 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] resource.caching = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.449188 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.449329 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.449470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.449606 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.449731 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.449856 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.449980 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.450103 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.450236 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.450360 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.450486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.450610 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.450734 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.450858 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.450995 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.451120 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.451245 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.451379 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.451505 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.451628 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] role.caching = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.451761 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.451886 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] role.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.452011 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.452134 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.452258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.452382 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.452516 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.452641 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.452766 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.452890 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.453014 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.453159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.453315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.453442 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.453569 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.453696 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.453821 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.454024 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.454154 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.454287 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.454414 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] resource.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.454540 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.454674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.454808 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.454945 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.455071 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.455198 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.455324 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.455450 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.455585 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.455734 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.455859 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.455985 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.456111 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.456236 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.456370 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.456498 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.456633 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.456762 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.456888 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.457014 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] role.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.457148 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.457294 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] role.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.457418 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.457546 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.457673 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.457802 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.457938 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.458067 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.458207 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.458333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.458467 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.458597 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.458734 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.458867 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.458997 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.459126 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.459255 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.459384 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.459520 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.459650 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.459779 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.459909 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.460038 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.460177 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.460314 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.460443 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.460572 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.460701 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.460829 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] token.caching = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.460967 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.461098 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.461098 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.461389 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.461510 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.461637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.461773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.461901 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.462040 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.462167 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.462245 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.462317 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.462398 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.462473 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.462545 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.462618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.462690 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.462762 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.462842 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.462916 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.462989 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.463061 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.463132 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.463204 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.463284 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.463358 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.463430 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.463502 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.463573 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.463653 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.463735 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.463817 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.463892 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.463964 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.464036 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.connection = **** {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.464108 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.464180 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.464260 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.464334 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.464406 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.464482 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.464554 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] token.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.464625 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.464705 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.464780 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.464851 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.464922 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.464994 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.465065 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.465065 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.465206 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.465295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.465369 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.465441 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.465512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.465596 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.465672 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.465744 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.465815 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.465886 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.465958 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.466037 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.466112 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.466183 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.466254 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.466326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.466397 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.466477 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.466552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.466624 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.466696 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.466767 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 09 20:13:22.466840 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.466840 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.466840 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.466840 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.connection = **** {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.466840 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.466840 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.467291 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.467291 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.467421 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.467657 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.467897 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.468140 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.468390 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.468631 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.468880 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.469120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.469395 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.469638 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.469887 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.470120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.470364 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.470603 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:13:22.470776 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 09 20:13:22.494403 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.494691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.495402 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.496584 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.496821 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.497052 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.503672 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.505656 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.532052 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.532278 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.534221 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.534459 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.587538 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.588225 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.636208 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.636434 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.637195 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.637458 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.793540 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.793772 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.793981 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.794197 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.794282 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.794413 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.794500 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.794626 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.794708 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.794884 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.795104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.795315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.817126 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.817234 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.817393 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.817556 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.817655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.817788 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.817872 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.818007 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.818089 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.818221 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.818305 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.818432 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.818516 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.818641 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.829444 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.829676 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.842404 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.842616 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.842745 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.842834 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.842912 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.843018 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.843139 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.843223 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.843360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.843453 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.843577 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.843660 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.843791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.844002 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:13:22.860597 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.860698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.860829 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 20:13:22.860909 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 20:13:22.861132 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.861216 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.861358 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 20:13:22.861460 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 20:13:22.861632 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.861716 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.861892 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.861983 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.862149 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.862232 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.862402 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.862484 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.871939 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.872091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.872214 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.872301 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.872483 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.872570 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.872761 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.872845 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.879995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:22.880090 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:22.882905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:22.883005 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:22.883188 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.883270 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.883468 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.883552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.883746 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.883840 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.884024 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.884106 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.884279 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.884362 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.884543 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.884627 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.884812 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.884893 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.885082 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.885719 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.885887 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.885969 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.886156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.886238 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.909256 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.909364 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.910719 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:22.910803 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:22.911000 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.911083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.915654 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:22.915742 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:22.915937 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.916019 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.916219 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.916313 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.922563 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:22.922657 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:22.922849 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.922933 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.928432 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:22.928531 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:22.928726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.928807 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.929011 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.929092 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.929316 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.929402 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.929594 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.929678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.929866 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.929963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.930133 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.930217 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.930396 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.930495 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.945826 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.945935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.946105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.946190 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.950497 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:22.950590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:22.953742 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.953829 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.954038 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.954122 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.958238 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:22.958340 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:22.961334 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:22.961483 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:22.965047 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.965269 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.965370 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.965567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.965650 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.965849 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.965933 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.966131 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.966217 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.966408 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.966492 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.966686 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.982472 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:22.982762 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.983028 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.983290 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.983788 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:22.984083 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.984541 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.984804 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.990141 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.991684 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.992058 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.993211 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.993511 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.993595 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.995124 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.995399 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.996383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:22.996675 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.996956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.997263 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.997654 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.998082 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:22.998367 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.998648 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.998929 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:22.999320 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.009087 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:23.009396 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.009678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.009956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.010860 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.011141 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.011265 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:23.011417 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.011547 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.011824 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.012098 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.012476 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.012755 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.013032 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.027299 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:23.028950 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:23.030220 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:23.031889 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:23.033095 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.034622 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.034792 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.036066 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:23.036282 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.036366 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.036736 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.037757 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:23.038041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.038433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.043134 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.044583 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:23.044791 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.046261 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:23.047459 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.047855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.048143 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.048422 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.049136 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.049561 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.049849 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.050126 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.056182 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:23.056470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.056748 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.057956 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:23.058243 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.058527 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.065307 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:23.065426 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:13:23.065594 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.065707 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.065875 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.065986 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.066148 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.066260 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.066420 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.066533 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.066691 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.066802 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.066964 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.067059 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.067250 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.067333 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.067516 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.067606 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.067795 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.067878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.068064 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.068146 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.068332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.068417 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.068611 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.068692 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.068925 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.069005 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:13:23.103774 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55588ebef9c0 pid: 81430 (default app) Sep 09 20:13:23.103876 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55588ebef9c0 pid: 81429 (default app) Sep 09 20:13:23.107028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971fb6c-e076-4539-9107-299450e316dd None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:23.107314 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971fb6c-e076-4539-9107-299450e316dd None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:23.107584 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971fb6c-e076-4539-9107-299450e316dd None None] PATH_INFO: `/v3` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:23.111708 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1/1] 10.222.0.26 () {50 vars in 791 bytes} [Fri Sep 9 20:13:23 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:29.870519 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a231d9-ce55-4b26-9434-23ca67c11dbe None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:29.871019 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a231d9-ce55-4b26-9434-23ca67c11dbe None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:29.871125 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a231d9-ce55-4b26-9434-23ca67c11dbe None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:29.875344 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1/2] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:13:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:13:29.881028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3702a-360c-4467-b352-2027c7abf569 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:29.881618 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3702a-360c-4467-b352-2027c7abf569 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:29.881618 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3702a-360c-4467-b352-2027c7abf569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:29.952073 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: INFO dbcounter [None req-3bb3702a-360c-4467-b352-2027c7abf569 None None] Registered counter for database keystone Sep 09 20:13:29.971133 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG oslo_db.sqlalchemy.engines [None req-3bb3702a-360c-4467-b352-2027c7abf569 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81430) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 09 20:13:29.977344 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writer thread running {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 09 20:13:30.155388 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG passlib.handlers.bcrypt [None req-3bb3702a-360c-4467-b352-2027c7abf569 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81430) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 09 20:13:30.155766 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG passlib.handlers.bcrypt [None req-3bb3702a-360c-4467-b352-2027c7abf569 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81430) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 09 20:13:30.213660 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-3bb3702a-360c-4467-b352-2027c7abf569 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:13:30.215257 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-3bb3702a-360c-4467-b352-2027c7abf569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:13:30.342345 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 2/3] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:13:29 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 09 20:13:30.830882 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a231d9-ce55-4b26-9434-23ca67c11dbe None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:30.899542 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: INFO dbcounter [None req-d5a231d9-ce55-4b26-9434-23ca67c11dbe None None] Registered counter for database keystone Sep 09 20:13:30.921539 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG oslo_db.sqlalchemy.engines [None req-d5a231d9-ce55-4b26-9434-23ca67c11dbe None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81429) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 09 20:13:30.927663 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writer thread running {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 09 20:13:31.048110 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a231d9-ce55-4b26-9434-23ca67c11dbe None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:31.052356 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4ea73-9aa3-4129-945b-72e07a2a8841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:31.052976 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4ea73-9aa3-4129-945b-72e07a2a8841 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:31.053273 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4ea73-9aa3-4129-945b-72e07a2a8841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:31.053555 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4ea73-9aa3-4129-945b-72e07a2a8841 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:31.061696 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc4ea73-9aa3-4129-945b-72e07a2a8841 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:31.113838 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6bc4ea73-9aa3-4129-945b-72e07a2a8841 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 09 20:13:31.114184 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc4ea73-9aa3-4129-945b-72e07a2a8841 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:31.123297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 2/4] 10.222.0.26 () {58 vars in 1324 bytes} [Fri Sep 9 20:13:30 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 20:13:31.129189 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb3702a-360c-4467-b352-2027c7abf569 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:31.146412 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb3702a-360c-4467-b352-2027c7abf569 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:31.151375 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea072e0-317f-4e81-a347-b08015ed8157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:31.152491 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea072e0-317f-4e81-a347-b08015ed8157 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:31.152773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea072e0-317f-4e81-a347-b08015ed8157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:31.153045 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea072e0-317f-4e81-a347-b08015ed8157 admin admin] PATH_INFO: `/v3/projects/e8a4d19e4885495c84cf29b80391fd52` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:31.154662 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea072e0-317f-4e81-a347-b08015ed8157 admin admin] RBAC: Authorizing `identity:get_project(project_id=e8a4d19e4885495c84cf29b80391fd52)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:31.207756 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5ea072e0-317f-4e81-a347-b08015ed8157 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 09 20:13:31.208103 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea072e0-317f-4e81-a347-b08015ed8157 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:31.212610 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 3/5] 10.222.0.26 () {56 vars in 1207 bytes} [Fri Sep 9 20:13:31 2022] GET /identity/v3/projects/e8a4d19e4885495c84cf29b80391fd52 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:13:33.312332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf6a82-23df-4adf-ba3b-f85aee490a35 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:33.312852 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf6a82-23df-4adf-ba3b-f85aee490a35 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:33.312933 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf6a82-23df-4adf-ba3b-f85aee490a35 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:33.316954 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 3/6] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:13:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:13:33.322328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ab2af-31b7-488b-ac05-9ab97136848e None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:33.322743 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ab2af-31b7-488b-ac05-9ab97136848e None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:33.322877 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ab2af-31b7-488b-ac05-9ab97136848e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:33.343016 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-ea2ab2af-31b7-488b-ac05-9ab97136848e None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:13:33.344503 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-ea2ab2af-31b7-488b-ac05-9ab97136848e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:13:33.352168 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 4/7] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:13:33 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 09 20:13:33.619766 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf6a82-23df-4adf-ba3b-f85aee490a35 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:33.631919 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf6a82-23df-4adf-ba3b-f85aee490a35 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:33.636676 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40a192-7295-415f-9783-ba9ab90348cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:33.637297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40a192-7295-415f-9783-ba9ab90348cb admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:33.637584 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40a192-7295-415f-9783-ba9ab90348cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:33.637859 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40a192-7295-415f-9783-ba9ab90348cb admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:33.645350 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a40a192-7295-415f-9783-ba9ab90348cb admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:33.647106 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a40a192-7295-415f-9783-ba9ab90348cb admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:33.656449 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 4/8] 10.222.0.26 () {58 vars in 1324 bytes} [Fri Sep 9 20:13:33 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 20:13:33.661569 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ab2af-31b7-488b-ac05-9ab97136848e None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:33.670569 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ab2af-31b7-488b-ac05-9ab97136848e None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:33.674905 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c326db66-a602-4ba3-a02f-1098d5d86a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:33.675530 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326db66-a602-4ba3-a02f-1098d5d86a82 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:33.675809 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326db66-a602-4ba3-a02f-1098d5d86a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:33.676081 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326db66-a602-4ba3-a02f-1098d5d86a82 admin admin] PATH_INFO: `/v3/users/b20f75fb724246318736ac6f2901b55a` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:33.677425 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c326db66-a602-4ba3-a02f-1098d5d86a82 admin admin] RBAC: Authorizing `identity:get_user(user_id=b20f75fb724246318736ac6f2901b55a)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:33.679222 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c326db66-a602-4ba3-a02f-1098d5d86a82 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:33.680727 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 5/9] 10.222.0.26 () {56 vars in 1198 bytes} [Fri Sep 9 20:13:33 2022] GET /identity/v3/users/b20f75fb724246318736ac6f2901b55a => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:35.872607 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a065c3d-7eff-473e-85ad-16e51e98b360 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:35.873136 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a065c3d-7eff-473e-85ad-16e51e98b360 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:35.873219 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a065c3d-7eff-473e-85ad-16e51e98b360 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:35.874519 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 5/10] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:13:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:13:35.879997 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567ad35-d75f-441c-abf1-d8d3246b4c17 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:35.880423 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567ad35-d75f-441c-abf1-d8d3246b4c17 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:35.880538 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567ad35-d75f-441c-abf1-d8d3246b4c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:35.898730 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-8567ad35-d75f-441c-abf1-d8d3246b4c17 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:13:35.899909 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-8567ad35-d75f-441c-abf1-d8d3246b4c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:13:35.954145 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c80d9de-3b04-4156-8261-17bdeee75727 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:35.954435 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c80d9de-3b04-4156-8261-17bdeee75727 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:35.954778 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c80d9de-3b04-4156-8261-17bdeee75727 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:35.956167 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 6/11] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:13:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:13:35.961122 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236f8e8-b4c5-47de-87d1-6f2eaf89f91d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:35.961429 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236f8e8-b4c5-47de-87d1-6f2eaf89f91d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:35.961702 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236f8e8-b4c5-47de-87d1-6f2eaf89f91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:35.981162 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 6/12] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:13:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:13:36.004299 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG passlib.handlers.bcrypt [None req-f236f8e8-b4c5-47de-87d1-6f2eaf89f91d None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81429) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 09 20:13:36.004953 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG passlib.handlers.bcrypt [None req-f236f8e8-b4c5-47de-87d1-6f2eaf89f91d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81429) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 09 20:13:36.054065 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f236f8e8-b4c5-47de-87d1-6f2eaf89f91d None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:13:36.055618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f236f8e8-b4c5-47de-87d1-6f2eaf89f91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:13:36.152254 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 7/13] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:13:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:13:36.252525 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567ad35-d75f-441c-abf1-d8d3246b4c17 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:36.278365 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:36.357060 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567ad35-d75f-441c-abf1-d8d3246b4c17 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:36.390522 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da506675-7146-41f4-8289-c0556a8b8ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:36.391155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da506675-7146-41f4-8289-c0556a8b8ecf None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:36.391464 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da506675-7146-41f4-8289-c0556a8b8ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:36.391743 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da506675-7146-41f4-8289-c0556a8b8ecf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:36.402601 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da506675-7146-41f4-8289-c0556a8b8ecf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:13:36.411343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-da506675-7146-41f4-8289-c0556a8b8ecf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:13:36.411343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:13:36.411343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:13:36.411343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:13:36.411343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:13:36.411343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:13:36.411343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:13:36.411343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:13:36.411343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:13:36.411343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:13:36.411343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:13:36.411343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:13:36.411343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:13:36.411343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:13:36.411343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:13:36.411343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:13:36.412455 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:13:36.412455 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:13:36.412455 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:13:36.412455 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:13:36.412455 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:13:36.412455 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:13:36.412455 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:13:36.412455 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:13:36.412455 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:13:36.412455 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:13:36.412455 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:13:36.412455 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:13:36.412455 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:13:36.412455 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:13:36.412455 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:13:36.412455 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:13:36.413534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:13:36.413534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:13:36.413534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:13:36.413534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:13:36.413534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:13:36.413534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:13:36.413534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:13:36.413534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:13:36.413534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:13:36.413534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:13:36.413534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:13:36.413534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:13:36.417383 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 7/14] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:13:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:13:36.418952 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f236f8e8-b4c5-47de-87d1-6f2eaf89f91d None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:36.419094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da506675-7146-41f4-8289-c0556a8b8ecf None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:36.500312 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:36.518966 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da506675-7146-41f4-8289-c0556a8b8ecf None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:36.520453 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f236f8e8-b4c5-47de-87d1-6f2eaf89f91d None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:36.522769 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:36.556313 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974194ef-2c33-4cf8-8a60-1b3e7781cfb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:36.557470 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974194ef-2c33-4cf8-8a60-1b3e7781cfb7 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:36.557750 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974194ef-2c33-4cf8-8a60-1b3e7781cfb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:36.558022 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974194ef-2c33-4cf8-8a60-1b3e7781cfb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:36.559064 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03407928-fe30-479d-97a9-d4833cd40224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:36.559927 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03407928-fe30-479d-97a9-d4833cd40224 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:36.560232 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03407928-fe30-479d-97a9-d4833cd40224 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:36.560512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03407928-fe30-479d-97a9-d4833cd40224 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:36.561376 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03407928-fe30-479d-97a9-d4833cd40224 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:13:36.561680 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03407928-fe30-479d-97a9-d4833cd40224 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:36.563194 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03407928-fe30-479d-97a9-d4833cd40224 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:36.574984 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 8/15] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:13:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:36.580828 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03407928-fe30-479d-97a9-d4833cd40224 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:36.624861 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974194ef-2c33-4cf8-8a60-1b3e7781cfb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:36.626355 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974194ef-2c33-4cf8-8a60-1b3e7781cfb7 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:36.683324 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03407928-fe30-479d-97a9-d4833cd40224 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:36.705532 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:36.717035 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f0145-b811-48de-8c61-ae3745b41773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:36.717922 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f0145-b811-48de-8c61-ae3745b41773 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:36.718210 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f0145-b811-48de-8c61-ae3745b41773 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:36.718489 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f0145-b811-48de-8c61-ae3745b41773 None admin] PATH_INFO: `/v3/users/b20f75fb724246318736ac6f2901b55a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:36.719969 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f0145-b811-48de-8c61-ae3745b41773 None admin] RBAC: Authorizing `identity:get_user(user_id=b20f75fb724246318736ac6f2901b55a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:36.721455 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f0145-b811-48de-8c61-ae3745b41773 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:36.723049 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 9/16] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:13:36 2022] GET /identity/v3/users/b20f75fb724246318736ac6f2901b55a => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:36.724323 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 8/17] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:13:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 20:13:36.728408 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f0145-b811-48de-8c61-ae3745b41773 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:36.730141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974194ef-2c33-4cf8-8a60-1b3e7781cfb7 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:36.763697 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:36.826179 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974194ef-2c33-4cf8-8a60-1b3e7781cfb7 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:36.829189 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f0145-b811-48de-8c61-ae3745b41773 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:36.860539 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ae55e-467a-4283-8ab4-4e28b6fbbdc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:36.861171 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ae55e-467a-4283-8ab4-4e28b6fbbdc7 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:36.861500 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ae55e-467a-4283-8ab4-4e28b6fbbdc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:36.861776 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ae55e-467a-4283-8ab4-4e28b6fbbdc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:36.863341 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78745e87-ec1b-4094-8a27-50aec5e3f6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:36.863972 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78745e87-ec1b-4094-8a27-50aec5e3f6e0 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:36.864252 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78745e87-ec1b-4094-8a27-50aec5e3f6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:36.864521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78745e87-ec1b-4094-8a27-50aec5e3f6e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:36.865645 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78745e87-ec1b-4094-8a27-50aec5e3f6e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:36.867079 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78745e87-ec1b-4094-8a27-50aec5e3f6e0 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:36.868619 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 10/18] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:13:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:36.872171 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ae55e-467a-4283-8ab4-4e28b6fbbdc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:36.873656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ae55e-467a-4283-8ab4-4e28b6fbbdc7 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:36.874288 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78745e87-ec1b-4094-8a27-50aec5e3f6e0 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:36.890429 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-8f5ae55e-467a-4283-8ab4-4e28b6fbbdc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:13:36.890429 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:13:36.890429 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:13:36.890429 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:13:36.890429 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:13:36.890429 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:13:36.890429 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:13:36.890429 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:13:36.890429 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:13:36.890429 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:13:36.890429 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:13:36.890429 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:13:36.890429 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:13:36.890429 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:13:36.890429 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:13:36.890429 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:13:36.891656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:13:36.891656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:13:36.891656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:13:36.891656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:13:36.891656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:13:36.891656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:13:36.891656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:13:36.891656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:13:36.891656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:13:36.891656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:13:36.891656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:13:36.891656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:13:36.891656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:13:36.891656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:13:36.891656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:13:36.891656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:13:36.891656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:13:36.892489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:13:36.892489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:13:36.892489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:13:36.892489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 9/19] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:13:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:13:36.897166 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ae55e-467a-4283-8ab4-4e28b6fbbdc7 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:36.931036 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:36.973873 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78745e87-ec1b-4094-8a27-50aec5e3f6e0 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:36.975538 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:36.995709 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ae55e-467a-4283-8ab4-4e28b6fbbdc7 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:37.009180 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116400b3-8ee6-4d0c-bcc7-6056d4b3a43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:37.009841 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116400b3-8ee6-4d0c-bcc7-6056d4b3a43c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:37.010119 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116400b3-8ee6-4d0c-bcc7-6056d4b3a43c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:37.010391 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116400b3-8ee6-4d0c-bcc7-6056d4b3a43c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:37.011118 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116400b3-8ee6-4d0c-bcc7-6056d4b3a43c None admin] RBAC: Adding query filter params (role.id=9121ba25a6d84756990f0c3a19e9fecb, scope.domain.id=default, user.id=b20f75fb724246318736ac6f2901b55a) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:13:37.011412 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116400b3-8ee6-4d0c-bcc7-6056d4b3a43c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:37.012843 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116400b3-8ee6-4d0c-bcc7-6056d4b3a43c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:37.020899 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 11/20] 10.222.0.26 () {56 vars in 1322 bytes} [Fri Sep 9 20:13:36 2022] GET /identity/v3/role_assignments?user.id=b20f75fb724246318736ac6f2901b55a&scope.domain.id=default&role.id=9121ba25a6d84756990f0c3a19e9fecb => generated 233 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:13:37.028645 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68067910-58e1-47fa-9883-145ee1a6fe3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:37.029308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68067910-58e1-47fa-9883-145ee1a6fe3c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:37.029592 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68067910-58e1-47fa-9883-145ee1a6fe3c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:37.029868 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68067910-58e1-47fa-9883-145ee1a6fe3c None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:37.030538 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68067910-58e1-47fa-9883-145ee1a6fe3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:13:37.030847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68067910-58e1-47fa-9883-145ee1a6fe3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:37.032278 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68067910-58e1-47fa-9883-145ee1a6fe3c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:37.052158 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 10/21] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:13:39.557465 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f9c13-7003-4d34-8ed8-adfc831fa6c0 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:39.558013 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f9c13-7003-4d34-8ed8-adfc831fa6c0 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:39.558013 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f9c13-7003-4d34-8ed8-adfc831fa6c0 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:39.559641 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 12/22] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:13:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:13:39.565006 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fbfb14-f819-4de6-9d77-dea988fcca0a None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:39.565537 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fbfb14-f819-4de6-9d77-dea988fcca0a None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:39.565709 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fbfb14-f819-4de6-9d77-dea988fcca0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:39.594423 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-a1fbfb14-f819-4de6-9d77-dea988fcca0a None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:13:39.595682 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a1fbfb14-f819-4de6-9d77-dea988fcca0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:13:39.682990 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 11/23] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:13:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:13:39.692960 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3f291-abf5-44c0-8aaa-d4026ab69558 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:39.693308 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3f291-abf5-44c0-8aaa-d4026ab69558 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:39.693571 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3f291-abf5-44c0-8aaa-d4026ab69558 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:39.695349 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 13/24] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:13:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:13:39.701149 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf86ff-ddb7-40aa-80b1-06e19b03f2eb None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:39.701772 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf86ff-ddb7-40aa-80b1-06e19b03f2eb None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:39.702275 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf86ff-ddb7-40aa-80b1-06e19b03f2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:39.730090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-73cf86ff-ddb7-40aa-80b1-06e19b03f2eb None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:13:39.731888 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-73cf86ff-ddb7-40aa-80b1-06e19b03f2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:13:39.757220 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:39.821444 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 12/25] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:13:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:13:39.943533 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507ce21-4bc2-4d6e-8147-78f74763ce79 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:39.943871 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507ce21-4bc2-4d6e-8147-78f74763ce79 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:39.944076 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507ce21-4bc2-4d6e-8147-78f74763ce79 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:39.945687 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 14/26] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:13:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:13:39.950890 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db135772-8f9c-49cc-a2ce-355798b4ceb0 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:39.951171 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db135772-8f9c-49cc-a2ce-355798b4ceb0 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:39.951443 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db135772-8f9c-49cc-a2ce-355798b4ceb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:39.978210 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-db135772-8f9c-49cc-a2ce-355798b4ceb0 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:13:39.979995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-db135772-8f9c-49cc-a2ce-355798b4ceb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:13:40.059591 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 13/27] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:13:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:13:40.112480 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507ce21-4bc2-4d6e-8147-78f74763ce79 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:40.238112 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d7af9-728b-460c-aa3c-742911cb2853 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:40.238444 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d7af9-728b-460c-aa3c-742911cb2853 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:40.238660 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d7af9-728b-460c-aa3c-742911cb2853 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:40.240022 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 14/28] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:13:40.241062 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507ce21-4bc2-4d6e-8147-78f74763ce79 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:40.249473 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece55dd-8897-4176-a3f7-497782864e91 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:40.249750 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece55dd-8897-4176-a3f7-497782864e91 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:40.250086 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece55dd-8897-4176-a3f7-497782864e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:40.274793 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-8ece55dd-8897-4176-a3f7-497782864e91 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:13:40.277172 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-8ece55dd-8897-4176-a3f7-497782864e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:13:40.308437 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8cf0e4-281f-41f3-9913-1c226832851f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:40.309073 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8cf0e4-281f-41f3-9913-1c226832851f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:40.309379 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8cf0e4-281f-41f3-9913-1c226832851f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:40.309650 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8cf0e4-281f-41f3-9913-1c226832851f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:40.333964 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8cf0e4-281f-41f3-9913-1c226832851f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:13:40.351867 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-8f8cf0e4-281f-41f3-9913-1c226832851f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:13:40.351867 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:13:40.351867 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:13:40.351867 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:13:40.351867 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:13:40.351867 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:13:40.351867 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:13:40.351867 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:13:40.351867 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:13:40.351867 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:13:40.351867 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:13:40.351867 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:13:40.351867 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:13:40.351867 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:13:40.351867 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:13:40.351867 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:13:40.352758 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:13:40.352758 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:13:40.352758 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:13:40.352758 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:13:40.352758 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:13:40.352758 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:13:40.352758 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:13:40.352758 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:13:40.352758 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:13:40.352758 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:13:40.352758 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:13:40.352758 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:13:40.352758 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:13:40.352758 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:13:40.352758 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:13:40.352758 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:13:40.354116 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:13:40.354116 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:13:40.354116 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:13:40.354116 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:13:40.354116 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:13:40.354116 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:13:40.354116 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:13:40.354116 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:13:40.354116 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:13:40.354116 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:13:40.354116 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:13:40.354116 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:13:40.355519 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 15/29] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:13:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:13:40.358244 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8cf0e4-281f-41f3-9913-1c226832851f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:40.400848 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 15/30] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:13:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:13:40.401434 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece55dd-8897-4176-a3f7-497782864e91 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:40.409597 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:40.430337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:40.516927 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8cf0e4-281f-41f3-9913-1c226832851f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:40.572308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece55dd-8897-4176-a3f7-497782864e91 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:40.609362 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf161517-fa8a-4e1e-aa5d-3d7bfe24a4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:40.609983 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf161517-fa8a-4e1e-aa5d-3d7bfe24a4db None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:40.610268 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf161517-fa8a-4e1e-aa5d-3d7bfe24a4db None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:40.610538 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf161517-fa8a-4e1e-aa5d-3d7bfe24a4db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:40.617895 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e81395d-6da3-440b-a2fd-18dc25efbf2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:40.618637 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81395d-6da3-440b-a2fd-18dc25efbf2a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:40.618929 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81395d-6da3-440b-a2fd-18dc25efbf2a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:40.619705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81395d-6da3-440b-a2fd-18dc25efbf2a None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:40.620448 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e81395d-6da3-440b-a2fd-18dc25efbf2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:13:40.620745 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e81395d-6da3-440b-a2fd-18dc25efbf2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:40.623143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e81395d-6da3-440b-a2fd-18dc25efbf2a None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:40.640426 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf161517-fa8a-4e1e-aa5d-3d7bfe24a4db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:40.641936 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf161517-fa8a-4e1e-aa5d-3d7bfe24a4db None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:40.656874 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 16/31] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:13:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:40.659872 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e81395d-6da3-440b-a2fd-18dc25efbf2a None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:40.675359 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-cf161517-fa8a-4e1e-aa5d-3d7bfe24a4db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:13:40.675359 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:13:40.675359 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:13:40.675359 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:13:40.675359 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:13:40.675359 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:13:40.675359 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:13:40.675359 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:13:40.675359 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:13:40.675359 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:13:40.675359 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:13:40.675359 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:13:40.675359 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:13:40.675359 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:13:40.675359 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:13:40.675359 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:13:40.676230 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:13:40.676230 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:13:40.676230 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:13:40.676230 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:13:40.676230 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:13:40.676230 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:13:40.676230 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:13:40.676230 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:13:40.676230 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:13:40.676230 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:13:40.676230 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:13:40.676230 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:13:40.676230 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:13:40.676230 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:13:40.676230 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:13:40.676230 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:13:40.676230 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:13:40.677047 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:13:40.677047 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:13:40.677047 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:13:40.681272 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 16/32] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:13:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:13:40.684928 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf161517-fa8a-4e1e-aa5d-3d7bfe24a4db None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:40.783007 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:40.800916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:40.807711 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf161517-fa8a-4e1e-aa5d-3d7bfe24a4db None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:40.837427 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e81395d-6da3-440b-a2fd-18dc25efbf2a None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:40.848478 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f341b9a9-801a-4ece-9428-eb46cceb03f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:40.849124 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341b9a9-801a-4ece-9428-eb46cceb03f9 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:40.849437 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341b9a9-801a-4ece-9428-eb46cceb03f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:40.849713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341b9a9-801a-4ece-9428-eb46cceb03f9 None admin] PATH_INFO: `/v3/users/b20f75fb724246318736ac6f2901b55a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:40.850841 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f341b9a9-801a-4ece-9428-eb46cceb03f9 None admin] RBAC: Authorizing `identity:get_user(user_id=b20f75fb724246318736ac6f2901b55a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:40.852283 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f341b9a9-801a-4ece-9428-eb46cceb03f9 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:40.854299 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 17/33] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:13:40 2022] GET /identity/v3/users/b20f75fb724246318736ac6f2901b55a => generated 247 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:40.855099 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f341b9a9-801a-4ece-9428-eb46cceb03f9 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:40.878390 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affaf2aa-4246-4403-b55e-7a91a3052111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:40.879277 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affaf2aa-4246-4403-b55e-7a91a3052111 None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:40.880005 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affaf2aa-4246-4403-b55e-7a91a3052111 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:40.880446 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affaf2aa-4246-4403-b55e-7a91a3052111 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:40.881300 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affaf2aa-4246-4403-b55e-7a91a3052111 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:40.882831 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affaf2aa-4246-4403-b55e-7a91a3052111 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:40.961159 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f341b9a9-801a-4ece-9428-eb46cceb03f9 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:40.996420 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1379aa-05b2-4fbc-908b-dd210b7c700a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:40.997094 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1379aa-05b2-4fbc-908b-dd210b7c700a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:40.997406 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1379aa-05b2-4fbc-908b-dd210b7c700a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:40.997682 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1379aa-05b2-4fbc-908b-dd210b7c700a None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:40.998338 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1379aa-05b2-4fbc-908b-dd210b7c700a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:13:40.998631 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1379aa-05b2-4fbc-908b-dd210b7c700a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:40.999986 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1379aa-05b2-4fbc-908b-dd210b7c700a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:41.017773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 17/34] 10.222.0.26 () {60 vars in 1131 bytes} [Fri Sep 9 20:13:40 2022] POST /identity/v3/roles => generated 219 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:13:41.019035 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affaf2aa-4246-4403-b55e-7a91a3052111 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:41.019588 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 18/35] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:13:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:41.020502 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1379aa-05b2-4fbc-908b-dd210b7c700a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:41.030187 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:41.137340 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1379aa-05b2-4fbc-908b-dd210b7c700a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:41.147657 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affaf2aa-4246-4403-b55e-7a91a3052111 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:41.188276 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec31e79-da03-4478-9c23-b4c632995f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:41.189565 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec31e79-da03-4478-9c23-b4c632995f52 None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:41.190291 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec31e79-da03-4478-9c23-b4c632995f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:41.190974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec31e79-da03-4478-9c23-b4c632995f52 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:41.192423 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec31e79-da03-4478-9c23-b4c632995f52 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:41.194216 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf86422-c58b-490c-a840-e7cb72baa133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:41.194832 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec31e79-da03-4478-9c23-b4c632995f52 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:41.194936 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf86422-c58b-490c-a840-e7cb72baa133 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:41.195142 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf86422-c58b-490c-a840-e7cb72baa133 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:41.195419 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf86422-c58b-490c-a840-e7cb72baa133 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:41.196657 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf86422-c58b-490c-a840-e7cb72baa133 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:41.198778 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf86422-c58b-490c-a840-e7cb72baa133 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:41.200758 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 19/36] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:13:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:41.201587 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:41.201693 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf86422-c58b-490c-a840-e7cb72baa133 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:41.218211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 18/37] 10.222.0.26 () {60 vars in 1131 bytes} [Fri Sep 9 20:13:41 2022] POST /identity/v3/roles => generated 225 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:13:41.219467 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec31e79-da03-4478-9c23-b4c632995f52 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:41.300326 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:41.330023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf86422-c58b-490c-a840-e7cb72baa133 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:41.370394 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263423f8-f481-4d81-b6d1-c793cc273d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:41.371241 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263423f8-f481-4d81-b6d1-c793cc273d51 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:41.371349 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263423f8-f481-4d81-b6d1-c793cc273d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:41.371608 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263423f8-f481-4d81-b6d1-c793cc273d51 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:41.372248 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263423f8-f481-4d81-b6d1-c793cc273d51 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:41.373762 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263423f8-f481-4d81-b6d1-c793cc273d51 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:41.423816 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec31e79-da03-4478-9c23-b4c632995f52 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:41.473284 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932d016-755a-4e11-91f6-72293d690956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:41.473950 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932d016-755a-4e11-91f6-72293d690956 None admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:41.474233 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932d016-755a-4e11-91f6-72293d690956 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:41.474511 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932d016-755a-4e11-91f6-72293d690956 None admin] PATH_INFO: `/v3/domains/default/users/b20f75fb724246318736ac6f2901b55a/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:41.476949 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3932d016-755a-4e11-91f6-72293d690956 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=b20f75fb724246318736ac6f2901b55a, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:41.479532 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3932d016-755a-4e11-91f6-72293d690956 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:41.494238 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 19/38] 10.222.0.26 () {58 vars in 1361 bytes} [Fri Sep 9 20:13:41 2022] PUT /identity/v3/domains/default/users/b20f75fb724246318736ac6f2901b55a/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:13:41.501317 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fbe9ed-1dbc-42d2-ad04-591a335e4baa None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:41.501317 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fbe9ed-1dbc-42d2-ad04-591a335e4baa None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:41.501317 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fbe9ed-1dbc-42d2-ad04-591a335e4baa None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:41.501317 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 20/39] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:13:41.508085 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2940d-df0e-4f80-a347-a91e9624f6ba None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:41.508534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2940d-df0e-4f80-a347-a91e9624f6ba None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:41.508910 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2940d-df0e-4f80-a347-a91e9624f6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:41.531444 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 20/40] 10.222.0.26 () {60 vars in 1141 bytes} [Fri Sep 9 20:13:41 2022] POST /identity/v3/projects => generated 301 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:13:41.648509 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-e7c2940d-df0e-4f80-a347-a91e9624f6ba None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:13:41.650796 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-e7c2940d-df0e-4f80-a347-a91e9624f6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:13:41.713752 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:41.737976 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 21/41] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:13:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:13:41.746972 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9efa4c-10ac-4d6e-9824-727c393a6a86 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:41.747472 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9efa4c-10ac-4d6e-9824-727c393a6a86 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:41.747685 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9efa4c-10ac-4d6e-9824-727c393a6a86 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:41.749139 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 21/42] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:13:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:13:41.763682 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5e0fa-cf4f-42df-8dae-95c7f6462e5a None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:41.764294 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5e0fa-cf4f-42df-8dae-95c7f6462e5a None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:41.764814 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5e0fa-cf4f-42df-8dae-95c7f6462e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:41.784359 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-39e5e0fa-cf4f-42df-8dae-95c7f6462e5a None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:13:41.785586 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-39e5e0fa-cf4f-42df-8dae-95c7f6462e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:13:41.859328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 22/43] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:13:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:13:42.014083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9efa4c-10ac-4d6e-9824-727c393a6a86 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:42.118605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9efa4c-10ac-4d6e-9824-727c393a6a86 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:42.124162 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5e0fa-cf4f-42df-8dae-95c7f6462e5a None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:42.150171 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:42.160066 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe6002-31f1-4685-9acb-747380c6cdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:42.160675 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe6002-31f1-4685-9acb-747380c6cdb7 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:42.160957 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe6002-31f1-4685-9acb-747380c6cdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:42.161228 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe6002-31f1-4685-9acb-747380c6cdb7 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:42.161941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbe6002-31f1-4685-9acb-747380c6cdb7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:42.163445 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbe6002-31f1-4685-9acb-747380c6cdb7 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:42.180055 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 22/44] 10.222.0.26 () {60 vars in 1131 bytes} [Fri Sep 9 20:13:42 2022] POST /identity/v3/roles => generated 223 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:13:42.235509 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5e0fa-cf4f-42df-8dae-95c7f6462e5a None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:42.255371 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:42.276239 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783821f8-a17b-4587-b608-73bb39c64362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:42.276810 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783821f8-a17b-4587-b608-73bb39c64362 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:42.277098 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783821f8-a17b-4587-b608-73bb39c64362 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:42.277408 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783821f8-a17b-4587-b608-73bb39c64362 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:42.278577 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783821f8-a17b-4587-b608-73bb39c64362 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:42.280063 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783821f8-a17b-4587-b608-73bb39c64362 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:42.281740 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 23/45] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:13:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:13:42.287321 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe6002-31f1-4685-9acb-747380c6cdb7 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:42.404610 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe6002-31f1-4685-9acb-747380c6cdb7 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:42.440917 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061827b3-9f2e-428c-9799-a65b0db3a72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:42.441564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061827b3-9f2e-428c-9799-a65b0db3a72c None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:42.441848 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061827b3-9f2e-428c-9799-a65b0db3a72c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:42.442121 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061827b3-9f2e-428c-9799-a65b0db3a72c None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:42.442767 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061827b3-9f2e-428c-9799-a65b0db3a72c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:42.444205 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061827b3-9f2e-428c-9799-a65b0db3a72c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:42.469111 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 23/46] 10.222.0.26 () {60 vars in 1141 bytes} [Fri Sep 9 20:13:42 2022] POST /identity/v3/projects => generated 312 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:13:43.767910 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bce14-bb01-435d-8467-8f2c1edb2f17 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:43.768413 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bce14-bb01-435d-8467-8f2c1edb2f17 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:43.768526 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bce14-bb01-435d-8467-8f2c1edb2f17 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:43.769818 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 24/47] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:13:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:13:43.774947 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f8834-44c0-417d-a620-a4037da48bf2 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:43.775846 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f8834-44c0-417d-a620-a4037da48bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:43.776017 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f8834-44c0-417d-a620-a4037da48bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:43.794672 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-759f8834-44c0-417d-a620-a4037da48bf2 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:13:43.796186 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-759f8834-44c0-417d-a620-a4037da48bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:13:43.824528 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:43.863782 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 24/48] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:13:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:13:44.135567 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2bce14-bb01-435d-8467-8f2c1edb2f17 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:44.233563 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2bce14-bb01-435d-8467-8f2c1edb2f17 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:44.267331 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c3071-efb3-44c7-9f46-83bbdf920da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:44.267958 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c3071-efb3-44c7-9f46-83bbdf920da8 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:44.268237 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c3071-efb3-44c7-9f46-83bbdf920da8 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:44.268512 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c3071-efb3-44c7-9f46-83bbdf920da8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:44.279251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c3071-efb3-44c7-9f46-83bbdf920da8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:13:44.284105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-f86c3071-efb3-44c7-9f46-83bbdf920da8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:13:44.284105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:13:44.284105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:13:44.284105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:13:44.284105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:13:44.284105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:13:44.284105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:13:44.284105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:13:44.284105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:13:44.284105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:13:44.284105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:13:44.284105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:13:44.284105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:13:44.284105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:13:44.284105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:13:44.284105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:13:44.284939 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:13:44.284939 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:13:44.284939 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:13:44.284939 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:13:44.284939 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:13:44.284939 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:13:44.284939 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:13:44.284939 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:13:44.284939 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:13:44.284939 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:13:44.284939 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:13:44.284939 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:13:44.284939 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:13:44.284939 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:13:44.284939 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:13:44.284939 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:13:44.285748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:13:44.285748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:13:44.285748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:13:44.285748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:13:44.285748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:13:44.285748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:13:44.285748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:13:44.285748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:13:44.285748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:13:44.285748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:13:44.285748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:13:44.285748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:13:44.285748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 25/49] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:13:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:13:44.292274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f8834-44c0-417d-a620-a4037da48bf2 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:44.490760 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f8834-44c0-417d-a620-a4037da48bf2 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:44.526753 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13196b3-984a-4c27-bab3-6b564914e892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:44.527396 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13196b3-984a-4c27-bab3-6b564914e892 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:44.527680 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13196b3-984a-4c27-bab3-6b564914e892 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:44.527960 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13196b3-984a-4c27-bab3-6b564914e892 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:44.528649 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13196b3-984a-4c27-bab3-6b564914e892 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:13:44.528955 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13196b3-984a-4c27-bab3-6b564914e892 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:44.532220 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13196b3-984a-4c27-bab3-6b564914e892 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:44.541434 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 25/50] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:13:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:44.547750 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c3071-efb3-44c7-9f46-83bbdf920da8 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:44.593952 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:44.630128 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6f301-9022-4c5d-b324-4bd4f7e222d4 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:44.630415 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6f301-9022-4c5d-b324-4bd4f7e222d4 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:44.630711 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6f301-9022-4c5d-b324-4bd4f7e222d4 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:44.632036 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 26/51] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:13:44.637554 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202e364-66c6-4d54-ad67-7960a4983c40 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:44.637841 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202e364-66c6-4d54-ad67-7960a4983c40 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:44.638117 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202e364-66c6-4d54-ad67-7960a4983c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:44.648405 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c3071-efb3-44c7-9f46-83bbdf920da8 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:44.656060 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-2202e364-66c6-4d54-ad67-7960a4983c40 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:13:44.657268 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2202e364-66c6-4d54-ad67-7960a4983c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:13:44.682729 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:44.686618 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2539ad7-cb60-4200-b2ae-3d2388ca0eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:44.687253 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2539ad7-cb60-4200-b2ae-3d2388ca0eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:44.687537 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2539ad7-cb60-4200-b2ae-3d2388ca0eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:44.687811 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2539ad7-cb60-4200-b2ae-3d2388ca0eb2 None admin] PATH_INFO: `/v3/users/b20f75fb724246318736ac6f2901b55a` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:44.688932 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2539ad7-cb60-4200-b2ae-3d2388ca0eb2 None admin] RBAC: Authorizing `identity:get_user(user_id=b20f75fb724246318736ac6f2901b55a)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:44.690411 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2539ad7-cb60-4200-b2ae-3d2388ca0eb2 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:44.692022 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 26/52] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:13:44 2022] GET /identity/v3/users/b20f75fb724246318736ac6f2901b55a => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:13:44.697810 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2539ad7-cb60-4200-b2ae-3d2388ca0eb2 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:44.725711 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 27/53] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:13:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:13:44.795983 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2539ad7-cb60-4200-b2ae-3d2388ca0eb2 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:44.809015 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:44.829622 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4822b1bb-82b1-48b5-96cc-2be64cb436e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:44.830258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822b1bb-82b1-48b5-96cc-2be64cb436e2 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:44.830542 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822b1bb-82b1-48b5-96cc-2be64cb436e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:44.830814 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822b1bb-82b1-48b5-96cc-2be64cb436e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:44.831923 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4822b1bb-82b1-48b5-96cc-2be64cb436e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:44.833393 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4822b1bb-82b1-48b5-96cc-2be64cb436e2 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:44.834938 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 27/54] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:13:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:13:44.840851 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202e364-66c6-4d54-ad67-7960a4983c40 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:44.937355 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202e364-66c6-4d54-ad67-7960a4983c40 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:44.970219 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de36f8-3050-47b8-859c-0f00c0bb00cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:44.970829 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de36f8-3050-47b8-859c-0f00c0bb00cc None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:44.971112 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de36f8-3050-47b8-859c-0f00c0bb00cc None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:44.971389 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de36f8-3050-47b8-859c-0f00c0bb00cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:44.972115 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0de36f8-3050-47b8-859c-0f00c0bb00cc None admin] RBAC: Adding query filter params (role.id=9121ba25a6d84756990f0c3a19e9fecb, scope.domain.id=default, user.id=b20f75fb724246318736ac6f2901b55a) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:13:44.972473 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0de36f8-3050-47b8-859c-0f00c0bb00cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:44.973929 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0de36f8-3050-47b8-859c-0f00c0bb00cc None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:44.979900 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 28/55] 10.222.0.26 () {56 vars in 1322 bytes} [Fri Sep 9 20:13:44 2022] GET /identity/v3/role_assignments?user.id=b20f75fb724246318736ac6f2901b55a&scope.domain.id=default&role.id=9121ba25a6d84756990f0c3a19e9fecb => generated 530 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:44.994268 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4822b1bb-82b1-48b5-96cc-2be64cb436e2 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:45.102315 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4822b1bb-82b1-48b5-96cc-2be64cb436e2 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:45.139084 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe91ccd-f0ec-4faf-8bab-40f255bd0417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:45.139733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe91ccd-f0ec-4faf-8bab-40f255bd0417 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:45.140017 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe91ccd-f0ec-4faf-8bab-40f255bd0417 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:45.140294 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe91ccd-f0ec-4faf-8bab-40f255bd0417 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:45.141453 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe91ccd-f0ec-4faf-8bab-40f255bd0417 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:45.142906 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe91ccd-f0ec-4faf-8bab-40f255bd0417 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:45.144659 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 28/56] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:13:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:45.150209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de36f8-3050-47b8-859c-0f00c0bb00cc None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:45.178107 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:45.252700 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de36f8-3050-47b8-859c-0f00c0bb00cc None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:45.285819 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897be143-d8c8-4e49-9809-85272ec2cc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:45.286422 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897be143-d8c8-4e49-9809-85272ec2cc4f None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:45.286700 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897be143-d8c8-4e49-9809-85272ec2cc4f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:45.286971 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897be143-d8c8-4e49-9809-85272ec2cc4f None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:45.287627 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897be143-d8c8-4e49-9809-85272ec2cc4f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:45.289059 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897be143-d8c8-4e49-9809-85272ec2cc4f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:45.313975 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 29/57] 10.222.0.26 () {60 vars in 1140 bytes} [Fri Sep 9 20:13:45 2022] POST /identity/v3/projects => generated 298 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:13:47.449116 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27393527-a6ad-4560-b0fc-fb05658d43b9 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:47.449667 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27393527-a6ad-4560-b0fc-fb05658d43b9 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:47.449753 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27393527-a6ad-4560-b0fc-fb05658d43b9 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:47.451072 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 29/58] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:13:47.456280 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f8645-c0b5-4142-a2a3-5988d3cc5802 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:47.456696 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f8645-c0b5-4142-a2a3-5988d3cc5802 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:47.456837 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f8645-c0b5-4142-a2a3-5988d3cc5802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:47.475173 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-260f8645-c0b5-4142-a2a3-5988d3cc5802 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:13:47.476364 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-260f8645-c0b5-4142-a2a3-5988d3cc5802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:13:47.541314 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 30/59] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:13:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:13:47.806524 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27393527-a6ad-4560-b0fc-fb05658d43b9 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:47.855695 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:47.905841 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27393527-a6ad-4560-b0fc-fb05658d43b9 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:47.937681 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaa4e77-9d6a-4b10-8e68-a657d10cc47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:47.938299 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa4e77-9d6a-4b10-8e68-a657d10cc47a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:47.938585 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa4e77-9d6a-4b10-8e68-a657d10cc47a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:47.938862 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa4e77-9d6a-4b10-8e68-a657d10cc47a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:47.939965 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaa4e77-9d6a-4b10-8e68-a657d10cc47a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:47.941431 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaa4e77-9d6a-4b10-8e68-a657d10cc47a None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:47.942973 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 30/60] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:13:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:47.948960 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f8645-c0b5-4142-a2a3-5988d3cc5802 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:47.957041 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:48.046840 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f8645-c0b5-4142-a2a3-5988d3cc5802 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:48.079808 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fe5ad3-8b5b-4aa7-94fd-9a7765113937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:48.080350 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe5ad3-8b5b-4aa7-94fd-9a7765113937 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:48.080633 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe5ad3-8b5b-4aa7-94fd-9a7765113937 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:48.080908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe5ad3-8b5b-4aa7-94fd-9a7765113937 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:48.081589 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fe5ad3-8b5b-4aa7-94fd-9a7765113937 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:48.083025 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fe5ad3-8b5b-4aa7-94fd-9a7765113937 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:48.108024 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 31/61] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Sep 9 20:13:47 2022] POST /identity/v3/users => generated 275 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:13:51.340823 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a174b5-c738-4d1a-9021-874867b77a3f None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:51.341852 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a174b5-c738-4d1a-9021-874867b77a3f None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:51.342330 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a174b5-c738-4d1a-9021-874867b77a3f None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:51.343891 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 31/62] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:13:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:13:51.349393 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7bbfc-7e89-4dbc-9b53-e4e318dcf64c None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:51.350207 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7bbfc-7e89-4dbc-9b53-e4e318dcf64c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:51.350680 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7bbfc-7e89-4dbc-9b53-e4e318dcf64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:51.378286 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-d0d7bbfc-7e89-4dbc-9b53-e4e318dcf64c None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:13:51.380092 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d0d7bbfc-7e89-4dbc-9b53-e4e318dcf64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:13:51.459662 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:51.475049 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 32/63] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:13:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:13:51.788672 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a174b5-c738-4d1a-9021-874867b77a3f None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:51.841857 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577524b-ee59-4264-96a1-af1b493fe3d1 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:51.842497 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577524b-ee59-4264-96a1-af1b493fe3d1 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:51.842966 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577524b-ee59-4264-96a1-af1b493fe3d1 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:51.844499 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 33/64] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:13:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:13:51.849599 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f93d8-290c-4d30-a2b6-a6ad6f5dee16 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:51.850122 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f93d8-290c-4d30-a2b6-a6ad6f5dee16 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:51.850609 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f93d8-290c-4d30-a2b6-a6ad6f5dee16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:51.874962 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-2d0f93d8-290c-4d30-a2b6-a6ad6f5dee16 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:13:51.876544 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2d0f93d8-290c-4d30-a2b6-a6ad6f5dee16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:13:51.938398 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a174b5-c738-4d1a-9021-874867b77a3f None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:51.957166 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 34/65] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:13:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:13:51.958772 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12450de7-d677-4055-ba73-b61778b9052e None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:51.959060 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12450de7-d677-4055-ba73-b61778b9052e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:51.959332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12450de7-d677-4055-ba73-b61778b9052e None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:51.961101 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 35/66] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:13:51.966824 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:51.967788 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75652b3e-4549-44e8-85b5-7d925ad277fe None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:51.968309 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75652b3e-4549-44e8-85b5-7d925ad277fe None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:51.968416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75652b3e-4549-44e8-85b5-7d925ad277fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:51.992717 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca450657-21b7-4605-81f0-a5154e2037b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:51.993841 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca450657-21b7-4605-81f0-a5154e2037b9 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:51.995001 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca450657-21b7-4605-81f0-a5154e2037b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:51.995510 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca450657-21b7-4605-81f0-a5154e2037b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:52.007754 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca450657-21b7-4605-81f0-a5154e2037b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:13:52.010358 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-75652b3e-4549-44e8-85b5-7d925ad277fe None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:13:52.011639 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-75652b3e-4549-44e8-85b5-7d925ad277fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:13:52.013795 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-ca450657-21b7-4605-81f0-a5154e2037b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:13:52.013795 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:13:52.013795 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:13:52.013795 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:13:52.013795 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:13:52.013795 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:13:52.013795 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:13:52.013795 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:13:52.013795 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:13:52.013795 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:13:52.013795 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:13:52.013795 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:13:52.013795 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:13:52.013795 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:13:52.013795 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:13:52.013795 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:13:52.014701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:13:52.014701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:13:52.014701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:13:52.014701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:13:52.014701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:13:52.014701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:13:52.014701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:13:52.014701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:13:52.014701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:13:52.014701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:13:52.014701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:13:52.014701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:13:52.014701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:13:52.014701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:13:52.014701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:13:52.014701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:13:52.015489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:13:52.015489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:13:52.015489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:13:52.015489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:13:52.015489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:13:52.015489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:13:52.015489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:13:52.015489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:13:52.015489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:13:52.015489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:13:52.015489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:13:52.015489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:13:52.023673 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 32/67] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:13:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:13:52.024694 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10e6de-542c-4288-a0dd-291d68ad702d None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:52.025584 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10e6de-542c-4288-a0dd-291d68ad702d None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:52.025756 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10e6de-542c-4288-a0dd-291d68ad702d None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:52.027707 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 33/68] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:13:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:13:52.028242 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10e6de-542c-4288-a0dd-291d68ad702d None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:52.098356 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 36/69] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:13:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:13:52.099663 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915d5f7-4338-44f2-ac75-bddb42861a5a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:52.099962 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915d5f7-4338-44f2-ac75-bddb42861a5a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:52.100242 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915d5f7-4338-44f2-ac75-bddb42861a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:52.120200 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-d915d5f7-4338-44f2-ac75-bddb42861a5a None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:13:52.121440 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d915d5f7-4338-44f2-ac75-bddb42861a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:13:52.153216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10e6de-542c-4288-a0dd-291d68ad702d None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:52.181916 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:52.198901 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14785f-454a-4a1d-96cd-4cea86f00ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:52.199461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14785f-454a-4a1d-96cd-4cea86f00ebe None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:52.199747 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14785f-454a-4a1d-96cd-4cea86f00ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:52.200028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14785f-454a-4a1d-96cd-4cea86f00ebe None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:52.200701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14785f-454a-4a1d-96cd-4cea86f00ebe None admin] RBAC: Adding query filter params (name=member) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:13:52.201102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14785f-454a-4a1d-96cd-4cea86f00ebe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:52.202810 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14785f-454a-4a1d-96cd-4cea86f00ebe None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:52.207018 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 37/70] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:13:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:13:52.208613 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d581956-78af-4c4f-9725-d3da1bc1b95c None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:52.209104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d581956-78af-4c4f-9725-d3da1bc1b95c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:52.209602 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d581956-78af-4c4f-9725-d3da1bc1b95c None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:52.211091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 38/71] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:13:52.218703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44239c60-a34d-4fbf-971b-16608fe83500 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:52.219144 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 34/72] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Sep 9 20:13:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:52.219526 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44239c60-a34d-4fbf-971b-16608fe83500 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:52.220033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44239c60-a34d-4fbf-971b-16608fe83500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:52.224729 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14785f-454a-4a1d-96cd-4cea86f00ebe None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:52.240227 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-44239c60-a34d-4fbf-971b-16608fe83500 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:13:52.241856 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-44239c60-a34d-4fbf-971b-16608fe83500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:13:52.275762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:52.330597 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 39/73] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:13:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:13:52.331916 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44239c60-a34d-4fbf-971b-16608fe83500 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:52.344790 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14785f-454a-4a1d-96cd-4cea86f00ebe None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:52.381889 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8027463c-3797-426f-837a-5d195241e877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:52.382969 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027463c-3797-426f-837a-5d195241e877 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:52.383474 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027463c-3797-426f-837a-5d195241e877 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:52.383910 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027463c-3797-426f-837a-5d195241e877 None admin] PATH_INFO: `/v3/users/ccfa3b4f784145efae478ef998af8590` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:52.400775 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8027463c-3797-426f-837a-5d195241e877 None admin] RBAC: Authorizing `identity:get_user(user_id=ccfa3b4f784145efae478ef998af8590)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:52.402691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8027463c-3797-426f-837a-5d195241e877 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:52.404788 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 35/74] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:13:52 2022] GET /identity/v3/users/ccfa3b4f784145efae478ef998af8590 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:13:52.414536 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8027463c-3797-426f-837a-5d195241e877 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:52.442403 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44239c60-a34d-4fbf-971b-16608fe83500 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:52.457603 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:52.481582 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475cdf2a-b0a4-4e9c-8021-4577037aa226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:52.482336 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475cdf2a-b0a4-4e9c-8021-4577037aa226 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:52.482834 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475cdf2a-b0a4-4e9c-8021-4577037aa226 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:52.483387 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475cdf2a-b0a4-4e9c-8021-4577037aa226 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:52.495394 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475cdf2a-b0a4-4e9c-8021-4577037aa226 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:13:52.500943 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-475cdf2a-b0a4-4e9c-8021-4577037aa226 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:13:52.500943 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:13:52.500943 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:13:52.500943 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:13:52.500943 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:13:52.500943 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:13:52.500943 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:13:52.500943 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:13:52.500943 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:13:52.500943 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:13:52.500943 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:13:52.500943 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:13:52.500943 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:13:52.500943 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:13:52.500943 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:13:52.500943 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:13:52.501908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:13:52.501908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:13:52.501908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:13:52.501908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:13:52.501908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:13:52.501908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:13:52.501908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:13:52.501908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:13:52.501908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:13:52.501908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:13:52.501908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:13:52.501908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:13:52.501908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:13:52.501908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:13:52.501908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:13:52.501908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:13:52.502670 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:13:52.502670 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:13:52.502670 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:13:52.502670 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:13:52.502670 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:13:52.502670 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:13:52.502670 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:13:52.502670 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:13:52.502670 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:13:52.502670 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:13:52.502670 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:13:52.502670 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:13:52.505206 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 40/75] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:13:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:13:52.506325 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475cdf2a-b0a4-4e9c-8021-4577037aa226 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:52.534801 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:52.542094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8027463c-3797-426f-837a-5d195241e877 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:52.576978 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c041bbd-80b1-4a7f-b20f-bf6c508ff795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:52.577613 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c041bbd-80b1-4a7f-b20f-bf6c508ff795 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:52.577896 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c041bbd-80b1-4a7f-b20f-bf6c508ff795 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:52.578548 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c041bbd-80b1-4a7f-b20f-bf6c508ff795 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:52.616881 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475cdf2a-b0a4-4e9c-8021-4577037aa226 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:52.649593 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c041bbd-80b1-4a7f-b20f-bf6c508ff795 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:52.651729 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c041bbd-80b1-4a7f-b20f-bf6c508ff795 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:52.654331 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7adac3-f643-4f98-9d34-79116a6965b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:52.654974 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7adac3-f643-4f98-9d34-79116a6965b0 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:52.655261 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7adac3-f643-4f98-9d34-79116a6965b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:52.655540 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7adac3-f643-4f98-9d34-79116a6965b0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:52.672102 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7adac3-f643-4f98-9d34-79116a6965b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:13:52.676996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-fb7adac3-f643-4f98-9d34-79116a6965b0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:13:52.676996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:13:52.676996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:13:52.676996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:13:52.676996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:13:52.676996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:13:52.676996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:13:52.676996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:13:52.676996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:13:52.676996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:13:52.676996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:13:52.676996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:13:52.676996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:13:52.676996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:13:52.676996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:13:52.676996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:13:52.677940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:13:52.677940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:13:52.677940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:13:52.677940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:13:52.677940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:13:52.677940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:13:52.677940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:13:52.677940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:13:52.677940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:13:52.677940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:13:52.677940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:13:52.677940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:13:52.677940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:13:52.677940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:13:52.677940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:13:52.677940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:13:52.678713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:13:52.678713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:13:52.678713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:13:52.678713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:13:52.678713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:13:52.678713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:13:52.678713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:13:52.678713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:13:52.678713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:13:52.678713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:13:52.678713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:13:52.678713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:13:52.678713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 41/76] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Sep 9 20:13:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:13:52.679703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7adac3-f643-4f98-9d34-79116a6965b0 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:52.711170 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:52.758838 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:52.766202 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 36/77] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:13:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:13:52.767027 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c041bbd-80b1-4a7f-b20f-bf6c508ff795 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:52.789672 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7adac3-f643-4f98-9d34-79116a6965b0 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:52.823752 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a50a2-9b4c-442d-96d0-8ab658c71720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:52.824362 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a50a2-9b4c-442d-96d0-8ab658c71720 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:52.824645 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a50a2-9b4c-442d-96d0-8ab658c71720 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:52.824923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a50a2-9b4c-442d-96d0-8ab658c71720 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:52.825644 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a50a2-9b4c-442d-96d0-8ab658c71720 None admin] RBAC: Adding query filter params (name=member) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:13:52.825953 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a50a2-9b4c-442d-96d0-8ab658c71720 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:52.827336 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a50a2-9b4c-442d-96d0-8ab658c71720 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:52.835271 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 42/78] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Sep 9 20:13:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:52.835852 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a50a2-9b4c-442d-96d0-8ab658c71720 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:52.871556 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c041bbd-80b1-4a7f-b20f-bf6c508ff795 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:52.905362 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fd774-2a99-46df-8538-8907280a2fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:52.906003 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fd774-2a99-46df-8538-8907280a2fa1 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:52.906288 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fd774-2a99-46df-8538-8907280a2fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:52.906572 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fd774-2a99-46df-8538-8907280a2fa1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:52.916568 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693fd774-2a99-46df-8538-8907280a2fa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:13:52.921425 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-693fd774-2a99-46df-8538-8907280a2fa1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:13:52.921425 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:13:52.921425 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:13:52.921425 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:13:52.921425 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:13:52.921425 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:13:52.921425 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:13:52.921425 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:13:52.921425 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:13:52.921425 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:13:52.921425 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:13:52.921425 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:13:52.921425 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:13:52.921425 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:13:52.921425 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:13:52.921425 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:13:52.922466 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:13:52.922466 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:13:52.922466 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:13:52.922466 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:13:52.922466 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:13:52.922466 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:13:52.922466 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:13:52.922466 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:13:52.922466 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:13:52.922466 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:13:52.922466 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:13:52.922466 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:13:52.922466 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:13:52.922466 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:13:52.922466 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:13:52.922466 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:13:52.923335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:13:52.923335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:13:52.923335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:13:52.923335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:13:52.923335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:13:52.923335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:13:52.923335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:13:52.923335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:13:52.923335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:13:52.923335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:13:52.923335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:13:52.923335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:13:52.923335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 37/79] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:13:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:13:52.923335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fd774-2a99-46df-8538-8907280a2fa1 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:52.932346 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:52.939373 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a50a2-9b4c-442d-96d0-8ab658c71720 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:52.972695 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a149b-3d2a-4daa-8fdf-db5c6beb6800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:52.973283 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a149b-3d2a-4daa-8fdf-db5c6beb6800 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:52.973574 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a149b-3d2a-4daa-8fdf-db5c6beb6800 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:52.973850 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a149b-3d2a-4daa-8fdf-db5c6beb6800 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:52.974985 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a149b-3d2a-4daa-8fdf-db5c6beb6800 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:52.976418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a149b-3d2a-4daa-8fdf-db5c6beb6800 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:52.977763 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:52.978134 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 43/80] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:13:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:52.982100 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a149b-3d2a-4daa-8fdf-db5c6beb6800 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:53.022078 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fd774-2a99-46df-8538-8907280a2fa1 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:53.054531 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8329ec-8951-406b-b11a-70eea6b2b343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:53.055126 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8329ec-8951-406b-b11a-70eea6b2b343 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:53.055404 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8329ec-8951-406b-b11a-70eea6b2b343 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:53.055675 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8329ec-8951-406b-b11a-70eea6b2b343 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:53.059988 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8329ec-8951-406b-b11a-70eea6b2b343 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:13:53.059988 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8329ec-8951-406b-b11a-70eea6b2b343 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:53.059988 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8329ec-8951-406b-b11a-70eea6b2b343 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:53.066009 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 38/81] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:13:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:53.066783 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8329ec-8951-406b-b11a-70eea6b2b343 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:53.082949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a149b-3d2a-4daa-8fdf-db5c6beb6800 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:53.120591 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a06014-83ec-4304-b71b-0229b0f46894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:53.121281 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a06014-83ec-4304-b71b-0229b0f46894 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:53.121573 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a06014-83ec-4304-b71b-0229b0f46894 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:53.121862 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a06014-83ec-4304-b71b-0229b0f46894 None admin] PATH_INFO: `/v3/projects/462d8ea458304464846aba18be1b8665` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:53.123006 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a06014-83ec-4304-b71b-0229b0f46894 None admin] RBAC: Authorizing `identity:get_project(project_id=462d8ea458304464846aba18be1b8665)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:53.124439 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a06014-83ec-4304-b71b-0229b0f46894 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:53.126324 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 44/82] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:13:52 2022] GET /identity/v3/projects/462d8ea458304464846aba18be1b8665 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:53.126888 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a06014-83ec-4304-b71b-0229b0f46894 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:53.161091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:53.176749 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8329ec-8951-406b-b11a-70eea6b2b343 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:53.194483 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:53.210971 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964a8eb4-6f24-442b-9fe1-d30cb54cf39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:53.211609 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a8eb4-6f24-442b-9fe1-d30cb54cf39e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:53.211888 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a8eb4-6f24-442b-9fe1-d30cb54cf39e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:53.212194 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a8eb4-6f24-442b-9fe1-d30cb54cf39e None admin] PATH_INFO: `/v3/users/ccfa3b4f784145efae478ef998af8590` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:53.213366 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964a8eb4-6f24-442b-9fe1-d30cb54cf39e None admin] RBAC: Authorizing `identity:get_user(user_id=ccfa3b4f784145efae478ef998af8590)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:53.214813 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964a8eb4-6f24-442b-9fe1-d30cb54cf39e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:53.216394 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 39/83] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:13:53 2022] GET /identity/v3/users/ccfa3b4f784145efae478ef998af8590 => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:53.217136 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964a8eb4-6f24-442b-9fe1-d30cb54cf39e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:53.235927 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a06014-83ec-4304-b71b-0229b0f46894 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:53.269435 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff11a2-11c1-4bb7-aa25-de4bc3597ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:53.270047 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff11a2-11c1-4bb7-aa25-de4bc3597ebb None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:53.270336 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff11a2-11c1-4bb7-aa25-de4bc3597ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:53.270616 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff11a2-11c1-4bb7-aa25-de4bc3597ebb None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:53.271292 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff11a2-11c1-4bb7-aa25-de4bc3597ebb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:13:53.271602 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff11a2-11c1-4bb7-aa25-de4bc3597ebb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:53.272973 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff11a2-11c1-4bb7-aa25-de4bc3597ebb None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:53.281390 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 45/84] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:13:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:53.282179 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff11a2-11c1-4bb7-aa25-de4bc3597ebb None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:53.318394 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964a8eb4-6f24-442b-9fe1-d30cb54cf39e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:53.351887 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e54aca-f65d-4a3d-b294-4bc2a1b7e369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:53.352489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e54aca-f65d-4a3d-b294-4bc2a1b7e369 None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:53.352757 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e54aca-f65d-4a3d-b294-4bc2a1b7e369 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:53.353666 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e54aca-f65d-4a3d-b294-4bc2a1b7e369 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:53.354309 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e54aca-f65d-4a3d-b294-4bc2a1b7e369 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:53.355745 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e54aca-f65d-4a3d-b294-4bc2a1b7e369 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:53.376391 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:53.382335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 40/85] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Sep 9 20:13:53 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:13:53.383138 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e54aca-f65d-4a3d-b294-4bc2a1b7e369 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:53.386348 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff11a2-11c1-4bb7-aa25-de4bc3597ebb None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:53.426379 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59253c12-21e6-4deb-a11e-243de44f6d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:53.427042 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59253c12-21e6-4deb-a11e-243de44f6d8a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:53.427328 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59253c12-21e6-4deb-a11e-243de44f6d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:53.427609 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59253c12-21e6-4deb-a11e-243de44f6d8a None admin] PATH_INFO: `/v3/users/ccfa3b4f784145efae478ef998af8590` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:53.428786 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59253c12-21e6-4deb-a11e-243de44f6d8a None admin] RBAC: Authorizing `identity:get_user(user_id=ccfa3b4f784145efae478ef998af8590)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:53.430357 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59253c12-21e6-4deb-a11e-243de44f6d8a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:53.431476 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:53.432136 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 46/86] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:13:53 2022] GET /identity/v3/users/ccfa3b4f784145efae478ef998af8590 => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:53.433034 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59253c12-21e6-4deb-a11e-243de44f6d8a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:53.505054 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e54aca-f65d-4a3d-b294-4bc2a1b7e369 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:53.542321 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77004ee-6102-4f67-9112-e92a5e1573d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:53.542957 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77004ee-6102-4f67-9112-e92a5e1573d4 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:53.543243 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77004ee-6102-4f67-9112-e92a5e1573d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:53.543527 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77004ee-6102-4f67-9112-e92a5e1573d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:53.544296 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77004ee-6102-4f67-9112-e92a5e1573d4 None admin] RBAC: Adding query filter params (role.id=63fc659c11144d369a3f3fa684e056ae, scope.project.id=462d8ea458304464846aba18be1b8665, user.id=ccfa3b4f784145efae478ef998af8590) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:13:53.544596 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77004ee-6102-4f67-9112-e92a5e1573d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:53.546118 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77004ee-6102-4f67-9112-e92a5e1573d4 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:53.553544 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 41/87] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:13:53 2022] GET /identity/v3/role_assignments?user.id=ccfa3b4f784145efae478ef998af8590&scope.project.id=462d8ea458304464846aba18be1b8665&role.id=63fc659c11144d369a3f3fa684e056ae => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:53.561699 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59253c12-21e6-4deb-a11e-243de44f6d8a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:53.562147 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77004ee-6102-4f67-9112-e92a5e1573d4 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:53.611178 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beeeaf9-105b-4d62-9ed9-95e9989e09b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:53.611928 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beeeaf9-105b-4d62-9ed9-95e9989e09b0 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:53.612219 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beeeaf9-105b-4d62-9ed9-95e9989e09b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:53.612494 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beeeaf9-105b-4d62-9ed9-95e9989e09b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:53.650907 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:53.708567 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77004ee-6102-4f67-9112-e92a5e1573d4 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:53.713857 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beeeaf9-105b-4d62-9ed9-95e9989e09b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:53.715362 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beeeaf9-105b-4d62-9ed9-95e9989e09b0 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:53.719627 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:53.753103 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877d62b-346c-4ab4-aa3f-42a36dda9fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:53.754227 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877d62b-346c-4ab4-aa3f-42a36dda9fe4 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:53.755019 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877d62b-346c-4ab4-aa3f-42a36dda9fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:53.756292 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877d62b-346c-4ab4-aa3f-42a36dda9fe4 None admin] PATH_INFO: `/v3/users/b20f75fb724246318736ac6f2901b55a` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:53.756561 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6877d62b-346c-4ab4-aa3f-42a36dda9fe4 None admin] RBAC: Authorizing `identity:get_user(user_id=b20f75fb724246318736ac6f2901b55a)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:53.758244 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6877d62b-346c-4ab4-aa3f-42a36dda9fe4 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:53.763964 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 42/88] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:13:53 2022] GET /identity/v3/users/b20f75fb724246318736ac6f2901b55a => generated 247 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:53.764616 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877d62b-346c-4ab4-aa3f-42a36dda9fe4 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:53.856312 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 47/89] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:13:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:13:53.857183 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beeeaf9-105b-4d62-9ed9-95e9989e09b0 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:53.896636 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877d62b-346c-4ab4-aa3f-42a36dda9fe4 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:53.932700 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7006d-eb4a-40af-b35f-64dff31c11cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:53.933742 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7006d-eb4a-40af-b35f-64dff31c11cb None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:53.934232 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7006d-eb4a-40af-b35f-64dff31c11cb None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:53.934709 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7006d-eb4a-40af-b35f-64dff31c11cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:53.934879 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:53.984382 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beeeaf9-105b-4d62-9ed9-95e9989e09b0 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:54.015969 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:54.048473 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f533fb-b425-4f6b-990a-4f5da33855a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:54.049100 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f533fb-b425-4f6b-990a-4f5da33855a4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:54.049413 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f533fb-b425-4f6b-990a-4f5da33855a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:54.049691 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f533fb-b425-4f6b-990a-4f5da33855a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:54.069476 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc7006d-eb4a-40af-b35f-64dff31c11cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:54.070958 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc7006d-eb4a-40af-b35f-64dff31c11cb None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:54.151111 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f533fb-b425-4f6b-990a-4f5da33855a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:54.152665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f533fb-b425-4f6b-990a-4f5da33855a4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:54.256542 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:54.271623 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 43/90] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:13:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:13:54.272245 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7006d-eb4a-40af-b35f-64dff31c11cb None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:54.333394 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 48/91] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:13:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:13:54.334089 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f533fb-b425-4f6b-990a-4f5da33855a4 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:54.418363 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:54.481323 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f533fb-b425-4f6b-990a-4f5da33855a4 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:54.518497 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45842f-acd9-4202-aa55-b82f498d65fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:54.519276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45842f-acd9-4202-aa55-b82f498d65fe None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:54.519572 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45842f-acd9-4202-aa55-b82f498d65fe None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:54.519855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45842f-acd9-4202-aa55-b82f498d65fe None admin] PATH_INFO: `/v3/projects/462d8ea458304464846aba18be1b8665` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:54.521008 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a45842f-acd9-4202-aa55-b82f498d65fe None admin] RBAC: Authorizing `identity:get_project(project_id=462d8ea458304464846aba18be1b8665)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:54.522040 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7006d-eb4a-40af-b35f-64dff31c11cb None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:54.522484 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a45842f-acd9-4202-aa55-b82f498d65fe None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:54.524321 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 49/92] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:13:54 2022] GET /identity/v3/projects/462d8ea458304464846aba18be1b8665 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:54.525508 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45842f-acd9-4202-aa55-b82f498d65fe None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:54.556514 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:54.601781 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094db050-4d04-43c9-8dad-20d3fc5d76e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:54.602428 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094db050-4d04-43c9-8dad-20d3fc5d76e7 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:54.602767 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094db050-4d04-43c9-8dad-20d3fc5d76e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:54.603034 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094db050-4d04-43c9-8dad-20d3fc5d76e7 None admin] PATH_INFO: `/v3/projects/13cefd4b72af472db0c98edb6a882154` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:54.607246 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094db050-4d04-43c9-8dad-20d3fc5d76e7 None admin] RBAC: Authorizing `identity:get_project(project_id=13cefd4b72af472db0c98edb6a882154)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:54.608711 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094db050-4d04-43c9-8dad-20d3fc5d76e7 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:54.611326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 44/93] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:13:54 2022] GET /identity/v3/projects/13cefd4b72af472db0c98edb6a882154 => generated 312 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:54.612100 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094db050-4d04-43c9-8dad-20d3fc5d76e7 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:54.697490 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45842f-acd9-4202-aa55-b82f498d65fe None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:54.775808 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18e7a4-93af-487d-bb71-4d4d72c245e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:54.776694 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18e7a4-93af-487d-bb71-4d4d72c245e1 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:54.777314 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18e7a4-93af-487d-bb71-4d4d72c245e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:54.777809 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18e7a4-93af-487d-bb71-4d4d72c245e1 None admin] PATH_INFO: `/v3/projects/462d8ea458304464846aba18be1b8665` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:54.779180 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b18e7a4-93af-487d-bb71-4d4d72c245e1 None admin] RBAC: Authorizing `identity:get_project(project_id=462d8ea458304464846aba18be1b8665)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:54.780919 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b18e7a4-93af-487d-bb71-4d4d72c245e1 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:54.783056 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 50/94] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:13:54 2022] GET /identity/v3/projects/462d8ea458304464846aba18be1b8665 => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:54.784035 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18e7a4-93af-487d-bb71-4d4d72c245e1 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:54.846080 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094db050-4d04-43c9-8dad-20d3fc5d76e7 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:54.897441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18e7a4-93af-487d-bb71-4d4d72c245e1 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:54.905594 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:54.936000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d49f00c-9249-434d-a141-92504b775b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:54.936951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49f00c-9249-434d-a141-92504b775b4a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:54.937491 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49f00c-9249-434d-a141-92504b775b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:54.938049 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49f00c-9249-434d-a141-92504b775b4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:54.939074 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d49f00c-9249-434d-a141-92504b775b4a None admin] RBAC: Adding query filter params (role.id=63fc659c11144d369a3f3fa684e056ae, scope.project.id=13cefd4b72af472db0c98edb6a882154, user.id=ccfa3b4f784145efae478ef998af8590) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:13:54.939586 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d49f00c-9249-434d-a141-92504b775b4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:54.941217 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d49f00c-9249-434d-a141-92504b775b4a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:54.948501 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 51/95] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:13:54 2022] GET /identity/v3/role_assignments?user.id=ccfa3b4f784145efae478ef998af8590&scope.project.id=13cefd4b72af472db0c98edb6a882154&role.id=63fc659c11144d369a3f3fa684e056ae => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:54.949590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d49f00c-9249-434d-a141-92504b775b4a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:54.949949 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed4ac7-d2f8-49b8-92bc-ef2140222ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:54.950588 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed4ac7-d2f8-49b8-92bc-ef2140222ce2 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:54.950876 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed4ac7-d2f8-49b8-92bc-ef2140222ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:54.951177 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed4ac7-d2f8-49b8-92bc-ef2140222ce2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:54.951914 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed4ac7-d2f8-49b8-92bc-ef2140222ce2 None admin] RBAC: Adding query filter params (role.id=d5528405b19b48a6bc4b7ff451332d8a, scope.project.id=462d8ea458304464846aba18be1b8665, user.id=ccfa3b4f784145efae478ef998af8590) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:13:54.952227 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed4ac7-d2f8-49b8-92bc-ef2140222ce2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:54.953896 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed4ac7-d2f8-49b8-92bc-ef2140222ce2 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:54.975094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 45/96] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:13:54 2022] GET /identity/v3/role_assignments?user.id=ccfa3b4f784145efae478ef998af8590&scope.project.id=462d8ea458304464846aba18be1b8665&role.id=d5528405b19b48a6bc4b7ff451332d8a => generated 259 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:55.066732 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d49f00c-9249-434d-a141-92504b775b4a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:55.108754 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd199697-779b-4e16-9a27-b37d77cb6523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:55.109928 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd199697-779b-4e16-9a27-b37d77cb6523 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:55.110423 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd199697-779b-4e16-9a27-b37d77cb6523 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:55.110899 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd199697-779b-4e16-9a27-b37d77cb6523 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:55.111872 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd199697-779b-4e16-9a27-b37d77cb6523 None admin] RBAC: Adding query filter params (role.id=9121ba25a6d84756990f0c3a19e9fecb, scope.project.id=462d8ea458304464846aba18be1b8665, user.id=b20f75fb724246318736ac6f2901b55a) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:13:55.112416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd199697-779b-4e16-9a27-b37d77cb6523 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:55.114183 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd199697-779b-4e16-9a27-b37d77cb6523 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:55.121478 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 52/97] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:13:54 2022] GET /identity/v3/role_assignments?user.id=b20f75fb724246318736ac6f2901b55a&scope.project.id=462d8ea458304464846aba18be1b8665&role.id=9121ba25a6d84756990f0c3a19e9fecb => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:56.069070 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab829d-ab35-431c-9cd4-bbfae94cdf62 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:56.070179 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab829d-ab35-431c-9cd4-bbfae94cdf62 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:56.070663 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab829d-ab35-431c-9cd4-bbfae94cdf62 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:56.072252 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 46/98] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:13:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:13:56.077759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51978df8-7d09-436e-b5a9-c7fe5ccb570c None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:56.078561 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51978df8-7d09-436e-b5a9-c7fe5ccb570c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:56.079038 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51978df8-7d09-436e-b5a9-c7fe5ccb570c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:56.105949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-51978df8-7d09-436e-b5a9-c7fe5ccb570c None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:13:56.107617 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-51978df8-7d09-436e-b5a9-c7fe5ccb570c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:13:56.153618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:56.244272 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 53/99] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:13:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:13:56.398676 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061061e-5b52-4bb7-bccd-92c6b4303d5d None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:56.399026 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061061e-5b52-4bb7-bccd-92c6b4303d5d None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:56.399202 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061061e-5b52-4bb7-bccd-92c6b4303d5d None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:56.401331 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 47/100] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:13:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:13:56.406714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dec15f-8655-45d9-b9c5-9e6da6e6a758 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:56.407004 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dec15f-8655-45d9-b9c5-9e6da6e6a758 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:56.407282 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dec15f-8655-45d9-b9c5-9e6da6e6a758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:56.450471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a2dec15f-8655-45d9-b9c5-9e6da6e6a758 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:13:56.451686 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a2dec15f-8655-45d9-b9c5-9e6da6e6a758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:13:56.563648 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 54/101] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:13:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:13:56.614882 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb252bf-a1be-413a-814e-0184e14fe401 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:56.615231 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb252bf-a1be-413a-814e-0184e14fe401 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:56.615543 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb252bf-a1be-413a-814e-0184e14fe401 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:56.617213 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 48/102] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:13:56.624051 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dec15f-8655-45d9-b9c5-9e6da6e6a758 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:56.641778 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde2972-cd21-4104-8df0-ebcee31765bf None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:56.641778 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde2972-cd21-4104-8df0-ebcee31765bf None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:56.641778 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde2972-cd21-4104-8df0-ebcee31765bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:56.664139 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:56.683082 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-6cde2972-cd21-4104-8df0-ebcee31765bf None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:13:56.684326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6cde2972-cd21-4104-8df0-ebcee31765bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:13:56.843885 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 49/103] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:13:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:13:56.853940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:56.882640 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dec15f-8655-45d9-b9c5-9e6da6e6a758 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:56.937586 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b9c645-9753-4f4a-aca6-569df573e115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:56.938143 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9c645-9753-4f4a-aca6-569df573e115 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:56.938426 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9c645-9753-4f4a-aca6-569df573e115 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:56.938702 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9c645-9753-4f4a-aca6-569df573e115 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:56.956901 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b9c645-9753-4f4a-aca6-569df573e115 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:13:56.961979 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-42b9c645-9753-4f4a-aca6-569df573e115 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:13:56.961979 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:13:56.961979 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:13:56.961979 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:13:56.961979 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:13:56.961979 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:13:56.961979 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:13:56.961979 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:13:56.961979 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:13:56.961979 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:13:56.961979 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:13:56.961979 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:13:56.961979 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:13:56.961979 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:13:56.961979 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:13:56.961979 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:13:56.963387 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:13:56.963387 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:13:56.963387 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:13:56.963387 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:13:56.963387 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:13:56.963387 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:13:56.963387 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:13:56.963387 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:13:56.963387 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:13:56.963387 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:13:56.963387 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:13:56.963387 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:13:56.963387 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:13:56.963387 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:13:56.963387 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:13:56.963387 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:13:56.964163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:13:56.964163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:13:56.964163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:13:56.964163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:13:56.964163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:13:56.964163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:13:56.964163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:13:56.964163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:13:56.964163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:13:56.964163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:13:56.964163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:13:56.964163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:13:56.966384 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 55/104] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:13:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:13:56.972261 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde2972-cd21-4104-8df0-ebcee31765bf None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:56.998977 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b9c645-9753-4f4a-aca6-569df573e115 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:57.136739 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde2972-cd21-4104-8df0-ebcee31765bf None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:57.193449 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b9c645-9753-4f4a-aca6-569df573e115 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:57.200817 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf92b1-aaee-4827-9723-3f3c51146889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:57.201784 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf92b1-aaee-4827-9723-3f3c51146889 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:57.202287 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf92b1-aaee-4827-9723-3f3c51146889 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:57.202779 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf92b1-aaee-4827-9723-3f3c51146889 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:57.203658 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf92b1-aaee-4827-9723-3f3c51146889 None admin] RBAC: Adding query filter params (name=member) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:13:57.204159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf92b1-aaee-4827-9723-3f3c51146889 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:57.205846 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf92b1-aaee-4827-9723-3f3c51146889 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:57.214344 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:57.227330 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 50/105] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Sep 9 20:13:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:57.233023 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf92b1-aaee-4827-9723-3f3c51146889 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:57.269441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585d49f-5842-4070-a9b7-8afc7a9d3520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:57.270100 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585d49f-5842-4070-a9b7-8afc7a9d3520 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:57.270389 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585d49f-5842-4070-a9b7-8afc7a9d3520 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:57.270670 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585d49f-5842-4070-a9b7-8afc7a9d3520 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:57.297675 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d585d49f-5842-4070-a9b7-8afc7a9d3520 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:13:57.302663 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-d585d49f-5842-4070-a9b7-8afc7a9d3520 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:13:57.302663 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:13:57.302663 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:13:57.302663 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:13:57.302663 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:13:57.302663 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:13:57.302663 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:13:57.302663 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:13:57.302663 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:13:57.302663 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:13:57.302663 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:13:57.302663 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:13:57.302663 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:13:57.302663 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:13:57.302663 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:13:57.302663 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:13:57.303549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:13:57.303549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:13:57.303549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:13:57.303549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:13:57.303549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:13:57.303549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:13:57.303549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:13:57.303549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:13:57.303549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:13:57.303549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:13:57.303549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:13:57.303549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:13:57.303549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:13:57.303549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:13:57.303549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:13:57.303549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:13:57.304354 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:13:57.304354 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:13:57.304354 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:13:57.304354 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:13:57.304354 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:13:57.304354 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:13:57.304354 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:13:57.304354 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:13:57.304354 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:13:57.304354 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:13:57.304354 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:13:57.304354 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:13:57.304354 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 56/106] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:13:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:13:57.310212 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585d49f-5842-4070-a9b7-8afc7a9d3520 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:57.323903 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:57.402005 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf92b1-aaee-4827-9723-3f3c51146889 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:57.444801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585d49f-5842-4070-a9b7-8afc7a9d3520 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:57.480431 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff5a198-15db-43a8-a992-7de014644c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:57.481071 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5a198-15db-43a8-a992-7de014644c4a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:57.481470 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5a198-15db-43a8-a992-7de014644c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:57.481919 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59047ad-aa1e-4be3-85e9-6efd699f447f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:57.482315 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5a198-15db-43a8-a992-7de014644c4a None admin] PATH_INFO: `/v3/users/ccfa3b4f784145efae478ef998af8590` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:57.482530 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59047ad-aa1e-4be3-85e9-6efd699f447f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:57.482819 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59047ad-aa1e-4be3-85e9-6efd699f447f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:57.483098 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59047ad-aa1e-4be3-85e9-6efd699f447f None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:57.483772 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59047ad-aa1e-4be3-85e9-6efd699f447f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:13:57.484081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59047ad-aa1e-4be3-85e9-6efd699f447f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:57.484668 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff5a198-15db-43a8-a992-7de014644c4a None admin] RBAC: Authorizing `identity:get_user(user_id=ccfa3b4f784145efae478ef998af8590)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:57.485475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59047ad-aa1e-4be3-85e9-6efd699f447f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:57.486617 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff5a198-15db-43a8-a992-7de014644c4a None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:57.488464 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 51/107] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:13:57 2022] GET /identity/v3/users/ccfa3b4f784145efae478ef998af8590 => generated 275 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:13:57.492843 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff5a198-15db-43a8-a992-7de014644c4a None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:57.494963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 57/108] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Sep 9 20:13:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:13:57.494963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59047ad-aa1e-4be3-85e9-6efd699f447f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:57.564602 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:57.601416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59047ad-aa1e-4be3-85e9-6efd699f447f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:57.636770 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336e6b3-27cf-4cd5-91d2-ec5ad00e09f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:57.637442 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336e6b3-27cf-4cd5-91d2-ec5ad00e09f7 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:57.637736 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336e6b3-27cf-4cd5-91d2-ec5ad00e09f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:57.638021 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336e6b3-27cf-4cd5-91d2-ec5ad00e09f7 None admin] PATH_INFO: `/v3/projects/462d8ea458304464846aba18be1b8665` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:57.639184 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1336e6b3-27cf-4cd5-91d2-ec5ad00e09f7 None admin] RBAC: Authorizing `identity:get_project(project_id=462d8ea458304464846aba18be1b8665)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:57.640877 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:57.641716 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1336e6b3-27cf-4cd5-91d2-ec5ad00e09f7 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:57.643378 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 58/109] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:13:57 2022] GET /identity/v3/projects/462d8ea458304464846aba18be1b8665 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:57.644145 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336e6b3-27cf-4cd5-91d2-ec5ad00e09f7 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:57.718908 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff5a198-15db-43a8-a992-7de014644c4a None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:57.763929 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336e6b3-27cf-4cd5-91d2-ec5ad00e09f7 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:57.800295 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c602a-25e0-4808-b36d-fae8fec679cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:57.800937 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c602a-25e0-4808-b36d-fae8fec679cc None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:57.801225 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c602a-25e0-4808-b36d-fae8fec679cc None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:57.801540 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c602a-25e0-4808-b36d-fae8fec679cc None admin] PATH_INFO: `/v3/users/13aeffc2d9264942acbb0e7d0e8ec830` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:57.803518 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:57.810010 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd3fed-def0-43fa-8567-92e4011a5a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:57.810669 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd3fed-def0-43fa-8567-92e4011a5a86 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:57.810950 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd3fed-def0-43fa-8567-92e4011a5a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:57.811231 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd3fed-def0-43fa-8567-92e4011a5a86 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:57.817988 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd3fed-def0-43fa-8567-92e4011a5a86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:57.819578 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd3fed-def0-43fa-8567-92e4011a5a86 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:57.819983 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c602a-25e0-4808-b36d-fae8fec679cc None admin] RBAC: Authorizing `identity:get_user(user_id=13aeffc2d9264942acbb0e7d0e8ec830)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:57.821274 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 52/110] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:13:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:57.821585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c602a-25e0-4808-b36d-fae8fec679cc None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:57.822253 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd3fed-def0-43fa-8567-92e4011a5a86 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:57.823264 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 59/111] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:13:57 2022] GET /identity/v3/users/13aeffc2d9264942acbb0e7d0e8ec830 => generated 289 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:57.830943 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c602a-25e0-4808-b36d-fae8fec679cc None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:58.021773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd3fed-def0-43fa-8567-92e4011a5a86 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:58.027347 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c602a-25e0-4808-b36d-fae8fec679cc None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:58.090153 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088a52e-aca3-4ba2-8bc0-d0bafc247cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:58.090824 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088a52e-aca3-4ba2-8bc0-d0bafc247cd0 None admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:58.091105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088a52e-aca3-4ba2-8bc0-d0bafc247cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:58.091376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088a52e-aca3-4ba2-8bc0-d0bafc247cd0 None admin] PATH_INFO: `/v3/projects/462d8ea458304464846aba18be1b8665/users/ccfa3b4f784145efae478ef998af8590/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:58.093961 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4088a52e-aca3-4ba2-8bc0-d0bafc247cd0 None admin] RBAC: Authorizing `identity:create_grant(project_id=462d8ea458304464846aba18be1b8665, user_id=ccfa3b4f784145efae478ef998af8590, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:58.095417 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4088a52e-aca3-4ba2-8bc0-d0bafc247cd0 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:58.098279 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:58.102216 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f3425-bc03-4b12-8db3-7fa6a298c6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:58.102952 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f3425-bc03-4b12-8db3-7fa6a298c6f0 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:58.103277 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f3425-bc03-4b12-8db3-7fa6a298c6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:58.103592 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f3425-bc03-4b12-8db3-7fa6a298c6f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:58.127996 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 53/112] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Sep 9 20:13:57 2022] PUT /identity/v3/projects/462d8ea458304464846aba18be1b8665/users/ccfa3b4f784145efae478ef998af8590/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:13:58.128816 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088a52e-aca3-4ba2-8bc0-d0bafc247cd0 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:58.242940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:58.267312 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f3425-bc03-4b12-8db3-7fa6a298c6f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:58.269262 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f3425-bc03-4b12-8db3-7fa6a298c6f0 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:58.347366 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088a52e-aca3-4ba2-8bc0-d0bafc247cd0 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:58.402594 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e552ec1-477c-4c23-a0a5-102837a97a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:58.403550 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e552ec1-477c-4c23-a0a5-102837a97a8a None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:58.404026 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e552ec1-477c-4c23-a0a5-102837a97a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:58.404494 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e552ec1-477c-4c23-a0a5-102837a97a8a None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:58.405438 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e552ec1-477c-4c23-a0a5-102837a97a8a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:58.407628 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e552ec1-477c-4c23-a0a5-102837a97a8a None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:58.437694 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 60/113] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:13:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:13:58.437694 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f6a97-c5f7-4bcc-85da-fdb008b750c4 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:58.437694 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f6a97-c5f7-4bcc-85da-fdb008b750c4 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:58.438017 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f6a97-c5f7-4bcc-85da-fdb008b750c4 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:58.445780 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 61/114] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:13:58 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:13:58.449274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f6a97-c5f7-4bcc-85da-fdb008b750c4 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:58.454983 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 54/115] 10.222.0.26 () {60 vars in 1141 bytes} [Fri Sep 9 20:13:58 2022] POST /identity/v3/projects => generated 302 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:13:58.460558 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52b94d-7a39-471b-b273-9aee3b43bdac None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:58.461708 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52b94d-7a39-471b-b273-9aee3b43bdac None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:58.462222 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52b94d-7a39-471b-b273-9aee3b43bdac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:58.500253 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-5c52b94d-7a39-471b-b273-9aee3b43bdac None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:13:58.502783 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-5c52b94d-7a39-471b-b273-9aee3b43bdac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:13:58.612424 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:58.638191 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:58.653543 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f6a97-c5f7-4bcc-85da-fdb008b750c4 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:58.657580 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 55/116] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:13:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:13:58.659213 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387397e-14f7-43b5-866d-b39cd55cb174 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:58.659647 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387397e-14f7-43b5-866d-b39cd55cb174 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:58.660060 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387397e-14f7-43b5-866d-b39cd55cb174 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:58.662841 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 56/117] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:13:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:13:58.664353 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccd9cf-c7e7-49f3-af83-a4a56fa30fa8 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:58.664836 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccd9cf-c7e7-49f3-af83-a4a56fa30fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:58.665438 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccd9cf-c7e7-49f3-af83-a4a56fa30fa8 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:58.667342 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 57/118] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:13:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:13:58.668597 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eadc30d-78cc-43a4-9cfe-44b6cd6cde36 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:58.669163 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eadc30d-78cc-43a4-9cfe-44b6cd6cde36 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:58.669680 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eadc30d-78cc-43a4-9cfe-44b6cd6cde36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:58.693826 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-6eadc30d-78cc-43a4-9cfe-44b6cd6cde36 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:13:58.695791 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6eadc30d-78cc-43a4-9cfe-44b6cd6cde36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:13:58.710800 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70114433-1050-4010-b5d3-3957c42d4f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:58.711881 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70114433-1050-4010-b5d3-3957c42d4f9a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:58.711881 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70114433-1050-4010-b5d3-3957c42d4f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:58.712709 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70114433-1050-4010-b5d3-3957c42d4f9a None admin] PATH_INFO: `/v3/projects/462d8ea458304464846aba18be1b8665` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:58.713943 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70114433-1050-4010-b5d3-3957c42d4f9a None admin] RBAC: Authorizing `identity:get_project(project_id=462d8ea458304464846aba18be1b8665)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:58.715508 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70114433-1050-4010-b5d3-3957c42d4f9a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:58.717345 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 62/119] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:13:58 2022] GET /identity/v3/projects/462d8ea458304464846aba18be1b8665 => generated 298 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:58.719276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13588119-f521-4fa7-a6ee-64564b6e1e5d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:58.719573 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13588119-f521-4fa7-a6ee-64564b6e1e5d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:58.719856 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13588119-f521-4fa7-a6ee-64564b6e1e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:58.741519 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-13588119-f521-4fa7-a6ee-64564b6e1e5d None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:13:58.742740 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-13588119-f521-4fa7-a6ee-64564b6e1e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:13:58.786679 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 58/120] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:13:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:13:58.788792 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eadc30d-78cc-43a4-9cfe-44b6cd6cde36 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:58.842764 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 63/121] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:13:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:13:58.943938 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eadc30d-78cc-43a4-9cfe-44b6cd6cde36 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:58.956368 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:58.993763 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad6546-49de-496a-80fa-d5d5b370c195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:58.994377 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad6546-49de-496a-80fa-d5d5b370c195 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:58.994658 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad6546-49de-496a-80fa-d5d5b370c195 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:58.994935 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad6546-49de-496a-80fa-d5d5b370c195 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:58.995676 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dad6546-49de-496a-80fa-d5d5b370c195 None admin] RBAC: Adding query filter params (role.id=3d90aea6ba6344ab9e478143d9be44fd, scope.project.id=462d8ea458304464846aba18be1b8665, user.id=13aeffc2d9264942acbb0e7d0e8ec830) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:13:58.995975 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dad6546-49de-496a-80fa-d5d5b370c195 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:58.997451 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dad6546-49de-496a-80fa-d5d5b370c195 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:59.010092 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 59/122] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:13:58 2022] GET /identity/v3/role_assignments?user.id=13aeffc2d9264942acbb0e7d0e8ec830&scope.project.id=462d8ea458304464846aba18be1b8665&role.id=3d90aea6ba6344ab9e478143d9be44fd => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:59.099113 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13588119-f521-4fa7-a6ee-64564b6e1e5d None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:59.100631 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad6546-49de-496a-80fa-d5d5b370c195 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:59.235059 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:59.245095 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad6546-49de-496a-80fa-d5d5b370c195 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:59.246883 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13588119-f521-4fa7-a6ee-64564b6e1e5d None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:59.288013 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1ee64-ea27-48fd-b03d-06a987063ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:59.288770 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1ee64-ea27-48fd-b03d-06a987063ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:59.289118 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1ee64-ea27-48fd-b03d-06a987063ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:59.289487 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1ee64-ea27-48fd-b03d-06a987063ea7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:59.295459 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535ca38-2942-4ecd-b4ff-73b94f9cf4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:59.296137 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535ca38-2942-4ecd-b4ff-73b94f9cf4b6 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:59.296425 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535ca38-2942-4ecd-b4ff-73b94f9cf4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:59.296704 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535ca38-2942-4ecd-b4ff-73b94f9cf4b6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:59.301552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc1ee64-ea27-48fd-b03d-06a987063ea7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:13:59.307745 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-afc1ee64-ea27-48fd-b03d-06a987063ea7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:13:59.307745 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:13:59.307745 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:13:59.307745 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:13:59.307745 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:13:59.307745 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:13:59.307745 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:13:59.307745 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:13:59.307745 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:13:59.307745 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:13:59.307745 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:13:59.307745 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:13:59.307745 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:13:59.307745 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:13:59.307745 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:13:59.307745 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:13:59.308759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:13:59.308759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:13:59.308759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:13:59.308759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:13:59.308759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:13:59.308759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:13:59.308759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:13:59.308759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:13:59.308759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:13:59.308759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:13:59.308759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:13:59.308759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:13:59.308759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:13:59.308759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:13:59.308759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:13:59.308759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:13:59.309612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:13:59.309612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:13:59.309612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:13:59.309612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:13:59.309612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:13:59.309612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:13:59.309612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:13:59.309612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:13:59.309612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:13:59.309612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:13:59.309612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:13:59.309612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:13:59.309612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 64/123] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:13:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:13:59.310772 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535ca38-2942-4ecd-b4ff-73b94f9cf4b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:13:59.310843 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1ee64-ea27-48fd-b03d-06a987063ea7 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:59.315079 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-1535ca38-2942-4ecd-b4ff-73b94f9cf4b6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:13:59.315079 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:13:59.315079 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:13:59.315079 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:13:59.315079 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:13:59.315079 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:13:59.315079 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:13:59.315079 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:13:59.315079 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:13:59.315079 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:13:59.315079 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:13:59.315079 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:13:59.315079 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:13:59.315079 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:13:59.315079 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:13:59.315079 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:13:59.317326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:13:59.317326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:13:59.317326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:13:59.317326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:13:59.317326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:13:59.317326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:13:59.317326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:13:59.317326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:13:59.317326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:13:59.317326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:13:59.317326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:13:59.317326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:13:59.317326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:13:59.317326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:13:59.317326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:13:59.317326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:13:59.319321 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:13:59.319321 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:13:59.319321 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:13:59.319321 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:13:59.319321 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:13:59.319321 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:13:59.319321 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:13:59.319321 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:13:59.319321 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:13:59.319321 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:13:59.319321 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:13:59.319321 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:13:59.319321 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 60/124] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Sep 9 20:13:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:13:59.319321 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535ca38-2942-4ecd-b4ff-73b94f9cf4b6 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:59.373715 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:59.428851 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1ee64-ea27-48fd-b03d-06a987063ea7 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:59.466241 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2e62e7-3b01-4b66-b00b-17e766d0f738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:59.466914 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e62e7-3b01-4b66-b00b-17e766d0f738 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:59.467202 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e62e7-3b01-4b66-b00b-17e766d0f738 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:59.467483 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e62e7-3b01-4b66-b00b-17e766d0f738 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:59.479512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2e62e7-3b01-4b66-b00b-17e766d0f738 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:13:59.485131 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-1f2e62e7-3b01-4b66-b00b-17e766d0f738 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:13:59.485131 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:13:59.485131 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:13:59.485131 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:13:59.485131 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:13:59.485131 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:13:59.485131 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:13:59.485131 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:13:59.485131 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:13:59.485131 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:13:59.485131 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:13:59.485131 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:13:59.485131 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:13:59.485131 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:13:59.485131 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:13:59.485131 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:13:59.486054 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:13:59.486054 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:13:59.486054 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:13:59.486054 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:13:59.486054 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:13:59.486054 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:13:59.486054 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:13:59.486054 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:13:59.486054 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:13:59.486054 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:13:59.486054 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:13:59.486054 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:13:59.486054 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:13:59.486054 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:13:59.486054 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:13:59.486054 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:13:59.486836 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:13:59.486836 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:13:59.486836 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:13:59.486836 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:13:59.486836 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:13:59.486836 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:13:59.486836 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:13:59.486836 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:13:59.486836 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:13:59.486836 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:13:59.486836 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:13:59.486836 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:13:59.486836 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 65/125] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:13:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:13:59.487469 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2e62e7-3b01-4b66-b00b-17e766d0f738 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:59.505610 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:59.507878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535ca38-2942-4ecd-b4ff-73b94f9cf4b6 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:59.601311 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f2b8f-0767-4c40-b7c4-1ba937a38dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:59.601935 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f2b8f-0767-4c40-b7c4-1ba937a38dec None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:59.602216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f2b8f-0767-4c40-b7c4-1ba937a38dec None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:59.602489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f2b8f-0767-4c40-b7c4-1ba937a38dec None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:59.603791 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f2b8f-0767-4c40-b7c4-1ba937a38dec None admin] RBAC: Adding query filter params (name=member) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:13:59.604092 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f2b8f-0767-4c40-b7c4-1ba937a38dec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:59.605499 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f2b8f-0767-4c40-b7c4-1ba937a38dec None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:59.615432 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2e62e7-3b01-4b66-b00b-17e766d0f738 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:59.639175 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 61/126] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Sep 9 20:13:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:59.640116 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f2b8f-0767-4c40-b7c4-1ba937a38dec None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:59.652457 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a4f99-b42f-4e01-add3-0da0552a8da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:59.653074 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a4f99-b42f-4e01-add3-0da0552a8da0 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:59.653404 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a4f99-b42f-4e01-add3-0da0552a8da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:59.653684 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a4f99-b42f-4e01-add3-0da0552a8da0 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:59.654362 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7a4f99-b42f-4e01-add3-0da0552a8da0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:13:59.654671 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7a4f99-b42f-4e01-add3-0da0552a8da0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:59.656067 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7a4f99-b42f-4e01-add3-0da0552a8da0 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:59.664696 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 66/127] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:13:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:59.666203 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a4f99-b42f-4e01-add3-0da0552a8da0 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:59.739102 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:59.771251 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a4f99-b42f-4e01-add3-0da0552a8da0 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:59.810999 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca3645-defd-4a22-9c89-4d04c36eb935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:59.811649 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca3645-defd-4a22-9c89-4d04c36eb935 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:59.811934 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca3645-defd-4a22-9c89-4d04c36eb935 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:59.812216 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca3645-defd-4a22-9c89-4d04c36eb935 None admin] PATH_INFO: `/v3/users/ccfa3b4f784145efae478ef998af8590` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:59.815892 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca3645-defd-4a22-9c89-4d04c36eb935 None admin] RBAC: Authorizing `identity:get_user(user_id=ccfa3b4f784145efae478ef998af8590)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:59.817381 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca3645-defd-4a22-9c89-4d04c36eb935 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:59.818995 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 67/128] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:13:59 2022] GET /identity/v3/users/ccfa3b4f784145efae478ef998af8590 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:59.819767 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca3645-defd-4a22-9c89-4d04c36eb935 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:59.886245 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:13:59.911166 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f2b8f-0767-4c40-b7c4-1ba937a38dec None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:59.936905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca3645-defd-4a22-9c89-4d04c36eb935 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:13:59.976088 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a9ed9-ecf5-4dad-b607-4a376b4f57ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:13:59.976974 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a9ed9-ecf5-4dad-b607-4a376b4f57ff None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:13:59.977435 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a9ed9-ecf5-4dad-b607-4a376b4f57ff None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:13:59.977905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a9ed9-ecf5-4dad-b607-4a376b4f57ff None admin] PATH_INFO: `/v3/users/ccfa3b4f784145efae478ef998af8590` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:13:59.979245 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40a9ed9-ecf5-4dad-b607-4a376b4f57ff None admin] RBAC: Authorizing `identity:get_user(user_id=ccfa3b4f784145efae478ef998af8590)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:13:59.980841 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40a9ed9-ecf5-4dad-b607-4a376b4f57ff None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:13:59.982638 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 68/129] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:13:59 2022] GET /identity/v3/users/ccfa3b4f784145efae478ef998af8590 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:13:59.983545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a9ed9-ecf5-4dad-b607-4a376b4f57ff None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:13:59.996002 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:00.021947 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda746b-954f-408f-b87e-82f8d3b65392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:00.022571 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda746b-954f-408f-b87e-82f8d3b65392 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:00.022854 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda746b-954f-408f-b87e-82f8d3b65392 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:00.023126 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda746b-954f-408f-b87e-82f8d3b65392 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:00.023805 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda746b-954f-408f-b87e-82f8d3b65392 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:00.024113 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda746b-954f-408f-b87e-82f8d3b65392 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:00.026074 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda746b-954f-408f-b87e-82f8d3b65392 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:00.055435 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 62/130] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:13:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:00.056978 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda746b-954f-408f-b87e-82f8d3b65392 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:00.094363 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a9ed9-ecf5-4dad-b607-4a376b4f57ff None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:00.134738 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731da62-5e88-46d0-8cbd-d1131f72031f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:00.135637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731da62-5e88-46d0-8cbd-d1131f72031f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:00.136064 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731da62-5e88-46d0-8cbd-d1131f72031f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:00.136482 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731da62-5e88-46d0-8cbd-d1131f72031f None admin] PATH_INFO: `/v3/projects/13cefd4b72af472db0c98edb6a882154` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:00.138806 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731da62-5e88-46d0-8cbd-d1131f72031f None admin] RBAC: Authorizing `identity:get_project(project_id=13cefd4b72af472db0c98edb6a882154)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:00.140395 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731da62-5e88-46d0-8cbd-d1131f72031f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:00.144137 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 69/131] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:13:59 2022] GET /identity/v3/projects/13cefd4b72af472db0c98edb6a882154 => generated 312 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:00.146267 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731da62-5e88-46d0-8cbd-d1131f72031f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:00.272180 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda746b-954f-408f-b87e-82f8d3b65392 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:00.307573 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:00.310014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5decd3a8-4e2a-4613-815e-15d8b55b517a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:00.310879 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5decd3a8-4e2a-4613-815e-15d8b55b517a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:00.311447 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5decd3a8-4e2a-4613-815e-15d8b55b517a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:00.311915 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5decd3a8-4e2a-4613-815e-15d8b55b517a None admin] PATH_INFO: `/v3/projects/462d8ea458304464846aba18be1b8665` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:00.313220 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5decd3a8-4e2a-4613-815e-15d8b55b517a None admin] RBAC: Authorizing `identity:get_project(project_id=462d8ea458304464846aba18be1b8665)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:00.314864 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5decd3a8-4e2a-4613-815e-15d8b55b517a None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:00.317687 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 63/132] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:14:00 2022] GET /identity/v3/projects/462d8ea458304464846aba18be1b8665 => generated 298 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:00.318832 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5decd3a8-4e2a-4613-815e-15d8b55b517a None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:00.347894 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731da62-5e88-46d0-8cbd-d1131f72031f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:00.367135 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:00.420734 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91fb73c-8015-4346-b559-58406e0fabcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:00.421805 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91fb73c-8015-4346-b559-58406e0fabcf None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:00.422317 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91fb73c-8015-4346-b559-58406e0fabcf None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:00.422789 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91fb73c-8015-4346-b559-58406e0fabcf None admin] PATH_INFO: `/v3/users/b20f75fb724246318736ac6f2901b55a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:00.424257 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91fb73c-8015-4346-b559-58406e0fabcf None admin] RBAC: Authorizing `identity:get_user(user_id=b20f75fb724246318736ac6f2901b55a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:00.425949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91fb73c-8015-4346-b559-58406e0fabcf None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:00.429141 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 70/133] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:14:00 2022] GET /identity/v3/users/b20f75fb724246318736ac6f2901b55a => generated 247 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:00.432278 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91fb73c-8015-4346-b559-58406e0fabcf None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:00.449294 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5decd3a8-4e2a-4613-815e-15d8b55b517a None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:00.512691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3806067-8b13-4552-8d58-9e1fcd873eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:00.513804 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3806067-8b13-4552-8d58-9e1fcd873eef None admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:00.514416 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3806067-8b13-4552-8d58-9e1fcd873eef None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:00.515936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3806067-8b13-4552-8d58-9e1fcd873eef None admin] PATH_INFO: `/v3/projects/13cefd4b72af472db0c98edb6a882154/users/ccfa3b4f784145efae478ef998af8590/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:00.519129 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3806067-8b13-4552-8d58-9e1fcd873eef None admin] RBAC: Authorizing `identity:create_grant(project_id=13cefd4b72af472db0c98edb6a882154, user_id=ccfa3b4f784145efae478ef998af8590, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:00.522085 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3806067-8b13-4552-8d58-9e1fcd873eef None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:00.547065 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 64/134] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Sep 9 20:14:00 2022] PUT /identity/v3/projects/13cefd4b72af472db0c98edb6a882154/users/ccfa3b4f784145efae478ef998af8590/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:14:00.548196 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3806067-8b13-4552-8d58-9e1fcd873eef None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:00.631741 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91fb73c-8015-4346-b559-58406e0fabcf None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:00.707681 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebaa855-488c-4c11-b79c-9e2669bf0fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:00.708750 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebaa855-488c-4c11-b79c-9e2669bf0fd4 None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:00.709429 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebaa855-488c-4c11-b79c-9e2669bf0fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:00.709934 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebaa855-488c-4c11-b79c-9e2669bf0fd4 None admin] PATH_INFO: `/v3/projects/462d8ea458304464846aba18be1b8665/users/ccfa3b4f784145efae478ef998af8590/roles/d5528405b19b48a6bc4b7ff451332d8a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:00.716193 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebaa855-488c-4c11-b79c-9e2669bf0fd4 None admin] RBAC: Authorizing `identity:create_grant(project_id=462d8ea458304464846aba18be1b8665, user_id=ccfa3b4f784145efae478ef998af8590, role_id=d5528405b19b48a6bc4b7ff451332d8a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:00.719559 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebaa855-488c-4c11-b79c-9e2669bf0fd4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:00.744365 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:00.758300 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 71/135] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Sep 9 20:14:00 2022] PUT /identity/v3/projects/462d8ea458304464846aba18be1b8665/users/ccfa3b4f784145efae478ef998af8590/roles/d5528405b19b48a6bc4b7ff451332d8a => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:14:00.781646 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3806067-8b13-4552-8d58-9e1fcd873eef None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:00.843097 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b940e750-028b-4b52-8ef5-68e66f3e68e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:00.843690 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940e750-028b-4b52-8ef5-68e66f3e68e1 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:00.843982 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940e750-028b-4b52-8ef5-68e66f3e68e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:00.844250 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940e750-028b-4b52-8ef5-68e66f3e68e1 None admin] PATH_INFO: `/v3/projects/462d8ea458304464846aba18be1b8665` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:00.845434 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b940e750-028b-4b52-8ef5-68e66f3e68e1 None admin] RBAC: Authorizing `identity:get_project(project_id=462d8ea458304464846aba18be1b8665)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:00.846959 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b940e750-028b-4b52-8ef5-68e66f3e68e1 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:00.848974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 65/136] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:14:00 2022] GET /identity/v3/projects/462d8ea458304464846aba18be1b8665 => generated 298 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:00.855083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebaa855-488c-4c11-b79c-9e2669bf0fd4 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:00.863227 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:00.906770 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07cd29-35cf-4f78-b909-06ec8ef3e851 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:00.907082 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07cd29-35cf-4f78-b909-06ec8ef3e851 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:00.907325 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07cd29-35cf-4f78-b909-06ec8ef3e851 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:00.908800 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 66/137] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:00.914491 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59797c57-2f35-43ec-8ad2-6aa3915d8ad2 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:00.914790 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59797c57-2f35-43ec-8ad2-6aa3915d8ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:00.915678 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59797c57-2f35-43ec-8ad2-6aa3915d8ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:00.944996 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-59797c57-2f35-43ec-8ad2-6aa3915d8ad2 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:00.946301 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-59797c57-2f35-43ec-8ad2-6aa3915d8ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:00.994551 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebaa855-488c-4c11-b79c-9e2669bf0fd4 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:01.036440 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 67/138] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:01.047130 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc7d808-302f-45b5-9ed6-a5bd0bdadf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:01.048136 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7d808-302f-45b5-9ed6-a5bd0bdadf7f None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:01.048684 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7d808-302f-45b5-9ed6-a5bd0bdadf7f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:01.049184 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7d808-302f-45b5-9ed6-a5bd0bdadf7f None admin] PATH_INFO: `/v3/projects/462d8ea458304464846aba18be1b8665/users/b20f75fb724246318736ac6f2901b55a/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:01.051574 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc7d808-302f-45b5-9ed6-a5bd0bdadf7f None admin] RBAC: Authorizing `identity:create_grant(project_id=462d8ea458304464846aba18be1b8665, user_id=b20f75fb724246318736ac6f2901b55a, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:01.053490 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc7d808-302f-45b5-9ed6-a5bd0bdadf7f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:01.067811 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 72/139] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Sep 9 20:14:00 2022] PUT /identity/v3/projects/462d8ea458304464846aba18be1b8665/users/b20f75fb724246318736ac6f2901b55a/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:14:01.323581 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59797c57-2f35-43ec-8ad2-6aa3915d8ad2 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:01.430497 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:01.454954 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59797c57-2f35-43ec-8ad2-6aa3915d8ad2 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:01.491389 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d9291-12b3-4651-886e-9a21dcae01c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:01.492038 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d9291-12b3-4651-886e-9a21dcae01c6 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:01.492320 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d9291-12b3-4651-886e-9a21dcae01c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:01.492598 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d9291-12b3-4651-886e-9a21dcae01c6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:01.508345 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1d9291-12b3-4651-886e-9a21dcae01c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:01.513369 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-8e1d9291-12b3-4651-886e-9a21dcae01c6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:14:01.513369 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:01.513369 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:01.513369 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:01.513369 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:01.513369 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:01.513369 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:01.513369 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:01.513369 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:01.513369 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:01.513369 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:01.513369 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:01.513369 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:01.513369 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:01.513369 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:01.513369 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:01.514343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:01.514343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:01.514343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:01.514343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:01.514343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:01.514343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:01.514343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:01.514343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:01.514343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:01.514343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:01.514343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:01.514343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:01.514343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:01.514343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:01.514343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:01.514343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:01.515363 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:01.515363 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:01.515363 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:01.515363 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:01.515363 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:01.515363 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:01.515363 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:01.515363 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:01.515363 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:01.515363 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:01.515363 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:14:01.515363 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:14:01.515363 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 68/140] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:14:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:01.520437 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc7d808-302f-45b5-9ed6-a5bd0bdadf7f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:01.616054 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:01.677529 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc7d808-302f-45b5-9ed6-a5bd0bdadf7f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:01.725973 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb017325-7d6f-48c0-8d05-6d24a9c84101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:01.726615 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb017325-7d6f-48c0-8d05-6d24a9c84101 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:01.726902 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb017325-7d6f-48c0-8d05-6d24a9c84101 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:01.727181 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb017325-7d6f-48c0-8d05-6d24a9c84101 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:01.727910 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb017325-7d6f-48c0-8d05-6d24a9c84101 None admin] RBAC: Adding query filter params (name=member) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:01.728216 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb017325-7d6f-48c0-8d05-6d24a9c84101 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:01.729615 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb017325-7d6f-48c0-8d05-6d24a9c84101 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:01.738593 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 73/141] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Sep 9 20:14:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:01.745369 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d9291-12b3-4651-886e-9a21dcae01c6 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:01.893928 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d9291-12b3-4651-886e-9a21dcae01c6 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:01.933046 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171e9c1-97ff-4642-aa6f-9924148e4320 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:01.933407 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171e9c1-97ff-4642-aa6f-9924148e4320 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:01.933633 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171e9c1-97ff-4642-aa6f-9924148e4320 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:01.935102 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 74/142] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:14:01.940744 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69adb8d5-f55d-49ff-8d5e-c35def70257a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:01.941023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69adb8d5-f55d-49ff-8d5e-c35def70257a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:01.941317 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69adb8d5-f55d-49ff-8d5e-c35def70257a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:01.948008 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff20ad-b7fb-4d6d-a569-482a57117103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:01.948966 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff20ad-b7fb-4d6d-a569-482a57117103 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:01.949492 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff20ad-b7fb-4d6d-a569-482a57117103 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:01.949970 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff20ad-b7fb-4d6d-a569-482a57117103 None admin] PATH_INFO: `/v3/users/ccfa3b4f784145efae478ef998af8590` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:01.951323 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ff20ad-b7fb-4d6d-a569-482a57117103 None admin] RBAC: Authorizing `identity:get_user(user_id=ccfa3b4f784145efae478ef998af8590)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:01.953079 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ff20ad-b7fb-4d6d-a569-482a57117103 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:01.954887 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 69/143] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:14:01 2022] GET /identity/v3/users/ccfa3b4f784145efae478ef998af8590 => generated 275 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:01.963404 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff20ad-b7fb-4d6d-a569-482a57117103 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:01.996869 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:02.003252 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-69adb8d5-f55d-49ff-8d5e-c35def70257a None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:02.007737 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-69adb8d5-f55d-49ff-8d5e-c35def70257a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:02.123373 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 75/144] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:02.125235 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13de74a-723e-46e5-993c-84c5e45d7941 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:02.126839 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13de74a-723e-46e5-993c-84c5e45d7941 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:02.127457 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13de74a-723e-46e5-993c-84c5e45d7941 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:02.136984 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 76/145] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:02.138737 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f95e7-4648-4450-894b-08f0c2942a67 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:02.139216 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f95e7-4648-4450-894b-08f0c2942a67 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:02.139707 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f95e7-4648-4450-894b-08f0c2942a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:02.176526 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-7a0f95e7-4648-4450-894b-08f0c2942a67 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:02.179325 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7a0f95e7-4648-4450-894b-08f0c2942a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:02.192716 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff20ad-b7fb-4d6d-a569-482a57117103 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:02.260102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a373b53-960b-43a7-b3be-b50483df6abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:02.261074 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a373b53-960b-43a7-b3be-b50483df6abd None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:02.261549 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a373b53-960b-43a7-b3be-b50483df6abd None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:02.262006 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a373b53-960b-43a7-b3be-b50483df6abd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:02.286856 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:02.332456 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 77/146] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:02.362405 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a373b53-960b-43a7-b3be-b50483df6abd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:02.364479 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a373b53-960b-43a7-b3be-b50483df6abd None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:02.379894 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:02.482652 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0f95e7-4648-4450-894b-08f0c2942a67 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:02.511232 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 70/147] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:14:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:14:02.516916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a373b53-960b-43a7-b3be-b50483df6abd None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:02.648744 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0f95e7-4648-4450-894b-08f0c2942a67 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:02.689210 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a373b53-960b-43a7-b3be-b50483df6abd None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:02.700973 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:02.723799 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdc5b7f-9a5e-4eff-92ad-395ac7008aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:02.724452 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc5b7f-9a5e-4eff-92ad-395ac7008aaa None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:02.724732 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc5b7f-9a5e-4eff-92ad-395ac7008aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:02.725005 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc5b7f-9a5e-4eff-92ad-395ac7008aaa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:02.743570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e7a773-6f42-4518-9f13-31f43c1ef3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:02.744124 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7a773-6f42-4518-9f13-31f43c1ef3d1 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:02.744459 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7a773-6f42-4518-9f13-31f43c1ef3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:02.744808 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7a773-6f42-4518-9f13-31f43c1ef3d1 None admin] PATH_INFO: `/v3/projects/462d8ea458304464846aba18be1b8665` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:02.750104 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e7a773-6f42-4518-9f13-31f43c1ef3d1 None admin] RBAC: Authorizing `identity:get_project(project_id=462d8ea458304464846aba18be1b8665)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:02.751680 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e7a773-6f42-4518-9f13-31f43c1ef3d1 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:02.754118 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 71/148] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:14:02 2022] GET /identity/v3/projects/462d8ea458304464846aba18be1b8665 => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:02.757964 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e7a773-6f42-4518-9f13-31f43c1ef3d1 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:02.759650 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdc5b7f-9a5e-4eff-92ad-395ac7008aaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:02.764855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-0fdc5b7f-9a5e-4eff-92ad-395ac7008aaa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:14:02.764855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:02.764855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:02.764855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:02.764855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:02.764855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:02.764855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:02.764855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:02.764855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:02.764855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:02.764855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:02.764855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:02.764855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:02.764855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:02.764855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:02.764855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:02.765813 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:02.765813 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:02.765813 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:02.765813 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:02.765813 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:02.765813 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:02.765813 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:02.765813 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:02.765813 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:02.765813 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:02.765813 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:02.765813 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:02.765813 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:02.765813 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:02.765813 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:02.765813 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:02.766807 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:02.766807 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:02.766807 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:02.766807 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:02.766807 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:02.766807 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:02.766807 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:02.766807 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:02.766807 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:02.766807 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:02.766807 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:14:02.766807 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:02.768779 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 78/149] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:14:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:14:02.769858 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdc5b7f-9a5e-4eff-92ad-395ac7008aaa None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:02.852664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:02.930847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e7a773-6f42-4518-9f13-31f43c1ef3d1 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:02.960280 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdc5b7f-9a5e-4eff-92ad-395ac7008aaa None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:02.994781 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c014a3fd-ac0a-4c30-8cd6-04c10187d69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:02.995386 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014a3fd-ac0a-4c30-8cd6-04c10187d69c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:02.995672 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014a3fd-ac0a-4c30-8cd6-04c10187d69c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:02.996021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014a3fd-ac0a-4c30-8cd6-04c10187d69c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:02.997412 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c014a3fd-ac0a-4c30-8cd6-04c10187d69c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:02.999970 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c014a3fd-ac0a-4c30-8cd6-04c10187d69c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:03.003953 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 72/150] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:14:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:03.005000 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c014a3fd-ac0a-4c30-8cd6-04c10187d69c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:03.027812 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd0c2a7-030a-4522-b0b6-efe69a492979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:03.028474 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0c2a7-030a-4522-b0b6-efe69a492979 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:03.028755 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0c2a7-030a-4522-b0b6-efe69a492979 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:03.029021 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0c2a7-030a-4522-b0b6-efe69a492979 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:03.030320 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd0c2a7-030a-4522-b0b6-efe69a492979 None admin] RBAC: Adding query filter params (role.id=63fc659c11144d369a3f3fa684e056ae, scope.project.id=462d8ea458304464846aba18be1b8665, user.id=ccfa3b4f784145efae478ef998af8590) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:03.030660 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd0c2a7-030a-4522-b0b6-efe69a492979 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:03.032261 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd0c2a7-030a-4522-b0b6-efe69a492979 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:03.052456 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 79/151] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:14:02 2022] GET /identity/v3/role_assignments?user.id=ccfa3b4f784145efae478ef998af8590&scope.project.id=462d8ea458304464846aba18be1b8665&role.id=63fc659c11144d369a3f3fa684e056ae => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:03.053438 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd0c2a7-030a-4522-b0b6-efe69a492979 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:03.108258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:03.182295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c014a3fd-ac0a-4c30-8cd6-04c10187d69c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:03.248143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc96c6-fb1b-4358-af7f-8ddd7c39904e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:03.249146 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc96c6-fb1b-4358-af7f-8ddd7c39904e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:03.249652 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc96c6-fb1b-4358-af7f-8ddd7c39904e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:03.250151 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc96c6-fb1b-4358-af7f-8ddd7c39904e None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:03.252003 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afc96c6-fb1b-4358-af7f-8ddd7c39904e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:03.252538 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afc96c6-fb1b-4358-af7f-8ddd7c39904e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:03.255276 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afc96c6-fb1b-4358-af7f-8ddd7c39904e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:03.282730 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 73/152] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Sep 9 20:14:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:03.283148 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd0c2a7-030a-4522-b0b6-efe69a492979 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:03.288734 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc96c6-fb1b-4358-af7f-8ddd7c39904e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:03.312109 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:03.345390 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c9d55-a2d4-4cbc-b521-2db16e65dfd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:03.346014 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c9d55-a2d4-4cbc-b521-2db16e65dfd5 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:03.346299 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c9d55-a2d4-4cbc-b521-2db16e65dfd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:03.346570 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c9d55-a2d4-4cbc-b521-2db16e65dfd5 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:03.347228 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2c9d55-a2d4-4cbc-b521-2db16e65dfd5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:03.348668 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2c9d55-a2d4-4cbc-b521-2db16e65dfd5 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:03.378124 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 80/153] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Sep 9 20:14:03 2022] POST /identity/v3/users => generated 283 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:14:03.445357 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc96c6-fb1b-4358-af7f-8ddd7c39904e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:03.452982 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:03.499675 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb187f-7497-4d1c-8a39-ef8c8e5255db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:03.500320 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb187f-7497-4d1c-8a39-ef8c8e5255db None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:03.500601 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb187f-7497-4d1c-8a39-ef8c8e5255db None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:03.500871 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb187f-7497-4d1c-8a39-ef8c8e5255db None admin] PATH_INFO: `/v3/users/13aeffc2d9264942acbb0e7d0e8ec830` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:03.502045 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb187f-7497-4d1c-8a39-ef8c8e5255db None admin] RBAC: Authorizing `identity:get_user(user_id=13aeffc2d9264942acbb0e7d0e8ec830)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:03.503512 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb187f-7497-4d1c-8a39-ef8c8e5255db None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:03.505641 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 74/154] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:14:03 2022] GET /identity/v3/users/13aeffc2d9264942acbb0e7d0e8ec830 => generated 289 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:03.511284 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c9d55-a2d4-4cbc-b521-2db16e65dfd5 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:03.651556 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c9d55-a2d4-4cbc-b521-2db16e65dfd5 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:03.697659 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6745cc-a8cd-4159-8ae5-3d297f0e34da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:03.698302 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6745cc-a8cd-4159-8ae5-3d297f0e34da None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:03.698580 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6745cc-a8cd-4159-8ae5-3d297f0e34da None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:03.698855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6745cc-a8cd-4159-8ae5-3d297f0e34da None admin] PATH_INFO: `/v3/projects/462d8ea458304464846aba18be1b8665` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:03.700193 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6745cc-a8cd-4159-8ae5-3d297f0e34da None admin] RBAC: Authorizing `identity:get_project(project_id=462d8ea458304464846aba18be1b8665)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:03.701659 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6745cc-a8cd-4159-8ae5-3d297f0e34da None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:03.703892 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 81/155] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:14:03 2022] GET /identity/v3/projects/462d8ea458304464846aba18be1b8665 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:03.709237 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb187f-7497-4d1c-8a39-ef8c8e5255db None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:03.811404 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055bf78-e562-41ef-95f9-c3a8814f710a None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:03.811588 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055bf78-e562-41ef-95f9-c3a8814f710a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:03.811858 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055bf78-e562-41ef-95f9-c3a8814f710a None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:03.813189 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 82/156] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:03.819288 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20e6b9-d94b-4d7f-bd1b-7a9b93325cc9 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:03.819567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20e6b9-d94b-4d7f-bd1b-7a9b93325cc9 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:03.819838 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20e6b9-d94b-4d7f-bd1b-7a9b93325cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:03.827755 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb187f-7497-4d1c-8a39-ef8c8e5255db None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:03.846937 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-1c20e6b9-d94b-4d7f-bd1b-7a9b93325cc9 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:03.848136 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1c20e6b9-d94b-4d7f-bd1b-7a9b93325cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:03.869514 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7bc6fd-23b2-438c-b5b5-07526c4ffc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:03.870533 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7bc6fd-23b2-438c-b5b5-07526c4ffc16 None admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:03.871036 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7bc6fd-23b2-438c-b5b5-07526c4ffc16 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:03.871618 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7bc6fd-23b2-438c-b5b5-07526c4ffc16 None admin] PATH_INFO: `/v3/projects/462d8ea458304464846aba18be1b8665/users/13aeffc2d9264942acbb0e7d0e8ec830/roles/3d90aea6ba6344ab9e478143d9be44fd` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:03.874476 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7bc6fd-23b2-438c-b5b5-07526c4ffc16 None admin] RBAC: Authorizing `identity:create_grant(project_id=462d8ea458304464846aba18be1b8665, user_id=13aeffc2d9264942acbb0e7d0e8ec830, role_id=3d90aea6ba6344ab9e478143d9be44fd)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:03.876174 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7bc6fd-23b2-438c-b5b5-07526c4ffc16 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:03.888647 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:03.897710 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 75/157] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Sep 9 20:14:03 2022] PUT /identity/v3/projects/462d8ea458304464846aba18be1b8665/users/13aeffc2d9264942acbb0e7d0e8ec830/roles/3d90aea6ba6344ab9e478143d9be44fd => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:14:03.950387 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 83/158] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:14:04.140780 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4b71a-4e3e-4883-a49d-de084cf05eec None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:04.141282 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4b71a-4e3e-4883-a49d-de084cf05eec None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:04.141707 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4b71a-4e3e-4883-a49d-de084cf05eec None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:04.143343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 76/159] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:04.149842 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f395d-07be-4dcb-bb24-e638ddfc9922 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:04.150314 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f395d-07be-4dcb-bb24-e638ddfc9922 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:04.150768 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f395d-07be-4dcb-bb24-e638ddfc9922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:04.171160 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-967f395d-07be-4dcb-bb24-e638ddfc9922 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:04.172342 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-967f395d-07be-4dcb-bb24-e638ddfc9922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:04.247491 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 84/160] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:04.260254 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b4b71a-4e3e-4883-a49d-de084cf05eec None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:04.317305 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:04.418532 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b4b71a-4e3e-4883-a49d-de084cf05eec None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:04.457822 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817b406-baa5-41de-a299-256bb4ecbe78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:04.458509 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817b406-baa5-41de-a299-256bb4ecbe78 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:04.458797 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817b406-baa5-41de-a299-256bb4ecbe78 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:04.459069 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817b406-baa5-41de-a299-256bb4ecbe78 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:04.470609 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2817b406-baa5-41de-a299-256bb4ecbe78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:04.474637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab6d59-78f7-4000-a9a6-10ca378c492c None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:04.474922 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab6d59-78f7-4000-a9a6-10ca378c492c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:04.475200 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab6d59-78f7-4000-a9a6-10ca378c492c None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:04.475989 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-2817b406-baa5-41de-a299-256bb4ecbe78 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:14:04.475989 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:04.475989 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:04.475989 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:04.475989 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:04.475989 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:04.475989 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:04.475989 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:04.475989 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:04.475989 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:04.475989 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:04.475989 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:04.475989 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:04.475989 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:04.475989 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:04.475989 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:04.476830 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:04.476830 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:04.476830 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:04.476830 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:04.476830 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:04.476830 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:04.476830 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:04.476830 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:04.476830 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:04.476830 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:04.476830 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:04.476830 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:04.476830 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:04.476830 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:04.476830 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:04.476830 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:04.477723 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:04.477723 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:04.477723 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:04.477723 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:04.477723 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:04.477723 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:04.477723 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:04.477723 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:04.477723 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:04.477723 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:04.477723 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:14:04.477723 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:14:04.478377 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 85/161] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:04.478451 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 77/162] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:14:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:04.484462 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9d770-13cd-4a21-bf9c-33b12fb66471 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:04.484973 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9d770-13cd-4a21-bf9c-33b12fb66471 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:04.485295 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9d770-13cd-4a21-bf9c-33b12fb66471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:04.485619 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817b406-baa5-41de-a299-256bb4ecbe78 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:04.521208 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-cba9d770-13cd-4a21-bf9c-33b12fb66471 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:04.522439 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-cba9d770-13cd-4a21-bf9c-33b12fb66471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:04.627688 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 86/163] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:04.628553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba9d770-13cd-4a21-bf9c-33b12fb66471 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:04.665093 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:04.679187 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817b406-baa5-41de-a299-256bb4ecbe78 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:04.680030 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:04.730845 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b764e4c1-a444-4a11-b2bb-4eed1c5ae787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:04.731565 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764e4c1-a444-4a11-b2bb-4eed1c5ae787 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:04.731788 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764e4c1-a444-4a11-b2bb-4eed1c5ae787 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:04.732057 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764e4c1-a444-4a11-b2bb-4eed1c5ae787 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:04.732834 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b764e4c1-a444-4a11-b2bb-4eed1c5ae787 None admin] RBAC: Adding query filter params (name=member) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:04.733133 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b764e4c1-a444-4a11-b2bb-4eed1c5ae787 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:04.734556 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b764e4c1-a444-4a11-b2bb-4eed1c5ae787 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:04.758969 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 78/164] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Sep 9 20:14:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:04.765554 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b764e4c1-a444-4a11-b2bb-4eed1c5ae787 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:04.831061 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba9d770-13cd-4a21-bf9c-33b12fb66471 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:04.876412 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34925ee1-4d79-4a56-bf6c-1b709b4326de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:04.877408 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34925ee1-4d79-4a56-bf6c-1b709b4326de None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:04.877698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34925ee1-4d79-4a56-bf6c-1b709b4326de None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:04.877977 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34925ee1-4d79-4a56-bf6c-1b709b4326de None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:04.901734 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34925ee1-4d79-4a56-bf6c-1b709b4326de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:04.906674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-34925ee1-4d79-4a56-bf6c-1b709b4326de None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:14:04.906674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:04.906674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:04.906674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:04.906674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:04.906674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:04.906674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:04.906674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:04.906674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:04.906674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:04.906674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:04.906674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:04.906674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:04.906674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:04.906674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:04.906674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:04.907567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:04.907567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:04.907567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:04.907567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:04.907567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:04.907567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:04.907567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:04.907567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:04.907567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:04.907567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:04.907567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:04.907567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:04.907567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:04.907567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:04.907567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:04.907567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:04.908375 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:04.908375 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:04.908375 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:04.908375 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:04.908375 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:04.908375 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:04.908375 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:04.908375 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:04.908375 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:04.908375 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:04.908375 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:14:04.908375 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:04.908375 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 87/165] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Sep 9 20:14:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:04.915052 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34925ee1-4d79-4a56-bf6c-1b709b4326de None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:04.921473 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b764e4c1-a444-4a11-b2bb-4eed1c5ae787 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:04.980954 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e485880-cc52-471f-8d41-7e8fef1f34a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:04.981666 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e485880-cc52-471f-8d41-7e8fef1f34a9 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:04.981946 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e485880-cc52-471f-8d41-7e8fef1f34a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:04.982217 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e485880-cc52-471f-8d41-7e8fef1f34a9 None admin] PATH_INFO: `/v3/users/ccfa3b4f784145efae478ef998af8590` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:04.983399 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e485880-cc52-471f-8d41-7e8fef1f34a9 None admin] RBAC: Authorizing `identity:get_user(user_id=ccfa3b4f784145efae478ef998af8590)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:04.984903 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e485880-cc52-471f-8d41-7e8fef1f34a9 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:04.986686 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 79/166] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:14:04 2022] GET /identity/v3/users/ccfa3b4f784145efae478ef998af8590 => generated 275 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:04.997495 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e485880-cc52-471f-8d41-7e8fef1f34a9 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:05.039672 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:05.091777 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:05.127214 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34925ee1-4d79-4a56-bf6c-1b709b4326de None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:05.174403 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e485880-cc52-471f-8d41-7e8fef1f34a9 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:05.196615 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80728c3-1afa-4239-89cf-da1c46617dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:05.197271 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80728c3-1afa-4239-89cf-da1c46617dbd None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:05.197561 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80728c3-1afa-4239-89cf-da1c46617dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:05.197838 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80728c3-1afa-4239-89cf-da1c46617dbd None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:05.198522 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80728c3-1afa-4239-89cf-da1c46617dbd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:05.198828 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80728c3-1afa-4239-89cf-da1c46617dbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:05.200224 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80728c3-1afa-4239-89cf-da1c46617dbd None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:05.218105 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 88/167] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:14:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:05.223580 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80728c3-1afa-4239-89cf-da1c46617dbd None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:05.231883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21dccf7-53f4-4e19-84ab-493764ed2a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:05.232498 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21dccf7-53f4-4e19-84ab-493764ed2a16 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:05.232777 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21dccf7-53f4-4e19-84ab-493764ed2a16 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:05.233048 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21dccf7-53f4-4e19-84ab-493764ed2a16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:05.382116 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21dccf7-53f4-4e19-84ab-493764ed2a16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:05.383992 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21dccf7-53f4-4e19-84ab-493764ed2a16 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:05.428990 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:05.432996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80728c3-1afa-4239-89cf-da1c46617dbd None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:05.484640 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1a387-08ed-4954-b584-b3fc4233ab7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:05.485200 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1a387-08ed-4954-b584-b3fc4233ab7f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:05.485505 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1a387-08ed-4954-b584-b3fc4233ab7f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:05.485778 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1a387-08ed-4954-b584-b3fc4233ab7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:05.506506 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1a387-08ed-4954-b584-b3fc4233ab7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:05.512634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-f6e1a387-08ed-4954-b584-b3fc4233ab7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:14:05.512634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:05.512634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:05.512634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:05.512634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:05.512634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:05.512634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:05.512634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:05.512634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:05.512634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:05.512634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:05.512634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:05.512634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:05.512634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:05.512634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:05.512634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:05.513665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:05.513665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:05.513665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:05.513665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:05.513665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:05.513665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:05.513665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:05.513665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:05.513665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:05.513665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:05.513665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:05.513665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:05.513665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:05.513665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:05.513665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:05.513665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:05.514471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:05.514471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:05.514471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:05.514471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:05.514471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:05.514471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:05.514471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:05.514471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:05.514471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:05.514471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:05.514471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:14:05.514471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:05.514471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 89/168] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:14:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:05.515145 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1a387-08ed-4954-b584-b3fc4233ab7f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:05.532704 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:05.611608 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 80/169] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:14:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 20:14:05.612482 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21dccf7-53f4-4e19-84ab-493764ed2a16 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:05.693688 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1a387-08ed-4954-b584-b3fc4233ab7f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:05.747193 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc9d70-f3ce-4f72-8fdd-f78a53000a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:05.747760 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc9d70-f3ce-4f72-8fdd-f78a53000a18 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:05.748038 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc9d70-f3ce-4f72-8fdd-f78a53000a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:05.748307 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc9d70-f3ce-4f72-8fdd-f78a53000a18 None admin] PATH_INFO: `/v3/users/ccfa3b4f784145efae478ef998af8590` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:05.753944 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fc9d70-f3ce-4f72-8fdd-f78a53000a18 None admin] RBAC: Authorizing `identity:get_user(user_id=ccfa3b4f784145efae478ef998af8590)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:05.755722 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fc9d70-f3ce-4f72-8fdd-f78a53000a18 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:05.757356 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 90/170] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:14:05 2022] GET /identity/v3/users/ccfa3b4f784145efae478ef998af8590 => generated 275 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:05.758122 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc9d70-f3ce-4f72-8fdd-f78a53000a18 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:05.895835 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:05.912979 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21dccf7-53f4-4e19-84ab-493764ed2a16 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:05.964085 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc9d70-f3ce-4f72-8fdd-f78a53000a18 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:05.965592 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:05.986762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229aafd-1c77-433d-b055-914d9896b7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:05.987396 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229aafd-1c77-433d-b055-914d9896b7bd None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:05.987680 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229aafd-1c77-433d-b055-914d9896b7bd None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:05.987954 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229aafd-1c77-433d-b055-914d9896b7bd None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:05.988647 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d229aafd-1c77-433d-b055-914d9896b7bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:05.988948 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d229aafd-1c77-433d-b055-914d9896b7bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:05.990516 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d229aafd-1c77-433d-b055-914d9896b7bd None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:06.005537 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 81/171] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:14:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:06.006413 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229aafd-1c77-433d-b055-914d9896b7bd None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:06.070502 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913516f6-edda-4908-87c2-4f115bc1021d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:06.071214 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913516f6-edda-4908-87c2-4f115bc1021d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:06.071504 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913516f6-edda-4908-87c2-4f115bc1021d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:06.071784 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913516f6-edda-4908-87c2-4f115bc1021d None admin] PATH_INFO: `/v3/projects/13cefd4b72af472db0c98edb6a882154` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:06.072924 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913516f6-edda-4908-87c2-4f115bc1021d None admin] RBAC: Authorizing `identity:get_project(project_id=13cefd4b72af472db0c98edb6a882154)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:06.074394 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913516f6-edda-4908-87c2-4f115bc1021d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:06.076031 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 91/172] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:14:05 2022] GET /identity/v3/projects/13cefd4b72af472db0c98edb6a882154 => generated 312 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:06.076793 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913516f6-edda-4908-87c2-4f115bc1021d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:06.203662 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229aafd-1c77-433d-b055-914d9896b7bd None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:06.284443 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691bc783-9817-49fa-a422-280ac5ec732a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:06.285445 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bc783-9817-49fa-a422-280ac5ec732a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:06.285927 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bc783-9817-49fa-a422-280ac5ec732a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:06.286453 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bc783-9817-49fa-a422-280ac5ec732a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:06.338638 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913516f6-edda-4908-87c2-4f115bc1021d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:06.405939 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:06.426916 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e570d6-1855-416b-bb75-a45595efe76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:06.427576 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e570d6-1855-416b-bb75-a45595efe76b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:06.427863 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e570d6-1855-416b-bb75-a45595efe76b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:06.428142 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e570d6-1855-416b-bb75-a45595efe76b None admin] PATH_INFO: `/v3/users/b20f75fb724246318736ac6f2901b55a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:06.429326 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e570d6-1855-416b-bb75-a45595efe76b None admin] RBAC: Authorizing `identity:get_user(user_id=b20f75fb724246318736ac6f2901b55a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:06.431337 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e570d6-1855-416b-bb75-a45595efe76b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:06.433124 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 92/173] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:14:06 2022] GET /identity/v3/users/b20f75fb724246318736ac6f2901b55a => generated 247 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:06.433726 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e570d6-1855-416b-bb75-a45595efe76b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:06.439610 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691bc783-9817-49fa-a422-280ac5ec732a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:06.441356 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691bc783-9817-49fa-a422-280ac5ec732a None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:06.465635 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:06.687401 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e570d6-1855-416b-bb75-a45595efe76b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:06.688347 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 82/174] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:14:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 682 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:14:06.689516 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691bc783-9817-49fa-a422-280ac5ec732a None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:06.796527 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95fa6f6-e73f-4ed3-9041-8ad04d9e993d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:06.797187 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95fa6f6-e73f-4ed3-9041-8ad04d9e993d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:06.797445 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95fa6f6-e73f-4ed3-9041-8ad04d9e993d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:06.798693 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95fa6f6-e73f-4ed3-9041-8ad04d9e993d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:06.799452 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95fa6f6-e73f-4ed3-9041-8ad04d9e993d None admin] RBAC: Adding query filter params (role.id=63fc659c11144d369a3f3fa684e056ae, scope.project.id=13cefd4b72af472db0c98edb6a882154, user.id=ccfa3b4f784145efae478ef998af8590) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:06.799763 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95fa6f6-e73f-4ed3-9041-8ad04d9e993d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:06.801192 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95fa6f6-e73f-4ed3-9041-8ad04d9e993d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:06.815560 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 93/175] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:14:06 2022] GET /identity/v3/role_assignments?user.id=ccfa3b4f784145efae478ef998af8590&scope.project.id=13cefd4b72af472db0c98edb6a882154&role.id=63fc659c11144d369a3f3fa684e056ae => generated 608 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:06.816198 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95fa6f6-e73f-4ed3-9041-8ad04d9e993d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:06.874518 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:06.909449 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691bc783-9817-49fa-a422-280ac5ec732a None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:06.952815 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:06.975399 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e4eb9b-8a95-45ce-956b-53316c82d9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:06.976019 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4eb9b-8a95-45ce-956b-53316c82d9e1 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:06.976295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4eb9b-8a95-45ce-956b-53316c82d9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:06.976565 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4eb9b-8a95-45ce-956b-53316c82d9e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:06.992798 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95fa6f6-e73f-4ed3-9041-8ad04d9e993d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:07.054861 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f45684-5785-4abc-93a9-f9b34af08f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:07.055951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f45684-5785-4abc-93a9-f9b34af08f43 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:07.056546 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f45684-5785-4abc-93a9-f9b34af08f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:07.058336 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f45684-5785-4abc-93a9-f9b34af08f43 None admin] PATH_INFO: `/v3/projects/462d8ea458304464846aba18be1b8665` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:07.065910 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f45684-5785-4abc-93a9-f9b34af08f43 None admin] RBAC: Authorizing `identity:get_project(project_id=462d8ea458304464846aba18be1b8665)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:07.067853 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f45684-5785-4abc-93a9-f9b34af08f43 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:07.069998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 94/176] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:14:06 2022] GET /identity/v3/projects/462d8ea458304464846aba18be1b8665 => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:07.076034 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f45684-5785-4abc-93a9-f9b34af08f43 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:07.126943 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e4eb9b-8a95-45ce-956b-53316c82d9e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:07.128825 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e4eb9b-8a95-45ce-956b-53316c82d9e1 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:07.232426 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f45684-5785-4abc-93a9-f9b34af08f43 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:07.264586 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:07.290578 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2212fc-6b38-4caf-b8d8-9a7da396361c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:07.291192 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2212fc-6b38-4caf-b8d8-9a7da396361c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:07.291470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2212fc-6b38-4caf-b8d8-9a7da396361c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:07.291740 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2212fc-6b38-4caf-b8d8-9a7da396361c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:07.292480 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2212fc-6b38-4caf-b8d8-9a7da396361c None admin] RBAC: Adding query filter params (role.id=d5528405b19b48a6bc4b7ff451332d8a, scope.project.id=462d8ea458304464846aba18be1b8665, user.id=ccfa3b4f784145efae478ef998af8590) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:07.292782 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2212fc-6b38-4caf-b8d8-9a7da396361c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:07.295562 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2212fc-6b38-4caf-b8d8-9a7da396361c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:07.301637 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 83/177] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:14:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:14:07.302757 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5b616-44ef-4c4c-9785-9939dfee12e4 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:07.303040 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5b616-44ef-4c4c-9785-9939dfee12e4 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:07.303307 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5b616-44ef-4c4c-9785-9939dfee12e4 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:07.310006 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 84/178] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:07.310803 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb5b616-44ef-4c4c-9785-9939dfee12e4 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:07.315511 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 95/179] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:14:07 2022] GET /identity/v3/role_assignments?user.id=ccfa3b4f784145efae478ef998af8590&scope.project.id=462d8ea458304464846aba18be1b8665&role.id=d5528405b19b48a6bc4b7ff451332d8a => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:07.322975 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d3158-6975-41c6-be2c-ea39a6ae1981 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:07.323582 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d3158-6975-41c6-be2c-ea39a6ae1981 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:07.324049 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d3158-6975-41c6-be2c-ea39a6ae1981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:07.350831 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:07.363613 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-7e2d3158-6975-41c6-be2c-ea39a6ae1981 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:07.365119 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7e2d3158-6975-41c6-be2c-ea39a6ae1981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:07.450087 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 96/180] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:07.451662 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beaebc3-0211-4bf2-81e7-98d114bee9db None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:07.452025 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beaebc3-0211-4bf2-81e7-98d114bee9db None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:07.452312 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beaebc3-0211-4bf2-81e7-98d114bee9db None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:07.458478 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 97/181] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:07.464434 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2c9ef5-ced5-404c-8d18-7a95f9fb7ed1 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:07.465111 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2c9ef5-ced5-404c-8d18-7a95f9fb7ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:07.465469 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2c9ef5-ced5-404c-8d18-7a95f9fb7ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:07.496109 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-7b2c9ef5-ced5-404c-8d18-7a95f9fb7ed1 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:07.497384 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7b2c9ef5-ced5-404c-8d18-7a95f9fb7ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:07.569210 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb5b616-44ef-4c4c-9785-9939dfee12e4 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:07.640435 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 98/182] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:07.641771 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39449545-aca0-4d76-a1da-f99c9f4e84d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:07.642420 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39449545-aca0-4d76-a1da-f99c9f4e84d5 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:07.642700 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39449545-aca0-4d76-a1da-f99c9f4e84d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:07.642975 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39449545-aca0-4d76-a1da-f99c9f4e84d5 None admin] PATH_INFO: `/v3/projects/462d8ea458304464846aba18be1b8665` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:07.644106 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39449545-aca0-4d76-a1da-f99c9f4e84d5 None admin] RBAC: Authorizing `identity:get_project(project_id=462d8ea458304464846aba18be1b8665)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:07.645582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39449545-aca0-4d76-a1da-f99c9f4e84d5 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:07.647207 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 85/183] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:14:07 2022] GET /identity/v3/projects/462d8ea458304464846aba18be1b8665 => generated 298 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:07.654311 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2c9ef5-ced5-404c-8d18-7a95f9fb7ed1 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:07.740575 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:07.792175 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2c9ef5-ced5-404c-8d18-7a95f9fb7ed1 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:07.841411 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc319b6-d1be-46c9-ac57-dbcf1e699538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:07.842050 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc319b6-d1be-46c9-ac57-dbcf1e699538 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:07.842342 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc319b6-d1be-46c9-ac57-dbcf1e699538 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:07.842626 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc319b6-d1be-46c9-ac57-dbcf1e699538 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:07.843415 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc319b6-d1be-46c9-ac57-dbcf1e699538 None admin] RBAC: Adding query filter params (role.id=9121ba25a6d84756990f0c3a19e9fecb, scope.project.id=462d8ea458304464846aba18be1b8665, user.id=b20f75fb724246318736ac6f2901b55a) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:07.843735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc319b6-d1be-46c9-ac57-dbcf1e699538 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:07.845160 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc319b6-d1be-46c9-ac57-dbcf1e699538 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:07.851563 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 99/184] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:14:07 2022] GET /identity/v3/role_assignments?user.id=b20f75fb724246318736ac6f2901b55a&scope.project.id=462d8ea458304464846aba18be1b8665&role.id=9121ba25a6d84756990f0c3a19e9fecb => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:07.871295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5130c-0e42-4dc0-b958-586ca029ad67 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:07.871624 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5130c-0e42-4dc0-b958-586ca029ad67 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:07.871858 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5130c-0e42-4dc0-b958-586ca029ad67 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:07.873207 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 86/185] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:07.878838 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97a8d4-4ad5-426b-a284-9ac6f423aedf None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:07.879136 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97a8d4-4ad5-426b-a284-9ac6f423aedf None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:07.879733 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97a8d4-4ad5-426b-a284-9ac6f423aedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:07.893547 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5130c-0e42-4dc0-b958-586ca029ad67 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:07.906421 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-fc97a8d4-4ad5-426b-a284-9ac6f423aedf None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:07.908010 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-fc97a8d4-4ad5-426b-a284-9ac6f423aedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:07.963127 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:08.004256 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 100/186] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:08.010176 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a57e5-3372-4a51-9e00-fdd5aaf72ea3 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:08.010565 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a57e5-3372-4a51-9e00-fdd5aaf72ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:08.010844 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a57e5-3372-4a51-9e00-fdd5aaf72ea3 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:08.013084 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 101/187] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:08.025595 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f371b-2895-4843-9741-7603eb5450d9 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:08.025956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f371b-2895-4843-9741-7603eb5450d9 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:08.026234 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f371b-2895-4843-9741-7603eb5450d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:08.049593 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5130c-0e42-4dc0-b958-586ca029ad67 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:08.061202 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0b2f371b-2895-4843-9741-7603eb5450d9 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:08.062440 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0b2f371b-2895-4843-9741-7603eb5450d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:08.105429 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82ac7e-d8b2-40c9-a5f0-e7cf306e0022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:08.106102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82ac7e-d8b2-40c9-a5f0-e7cf306e0022 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:08.106387 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82ac7e-d8b2-40c9-a5f0-e7cf306e0022 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:08.106668 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82ac7e-d8b2-40c9-a5f0-e7cf306e0022 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:08.123439 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d82ac7e-d8b2-40c9-a5f0-e7cf306e0022 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:08.129069 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-5d82ac7e-d8b2-40c9-a5f0-e7cf306e0022 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:14:08.129069 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:08.129069 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:08.129069 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:08.129069 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:08.129069 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:08.129069 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:08.129069 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:08.129069 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:08.129069 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:08.129069 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:08.129069 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:08.129069 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:08.129069 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:08.129069 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:08.129069 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:08.130262 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:08.130262 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:08.130262 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:08.130262 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:08.130262 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:08.130262 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:08.130262 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:08.130262 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:08.130262 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:08.130262 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:08.130262 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:08.130262 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:08.130262 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:08.130262 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:08.130262 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:08.130262 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:08.131056 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:08.131056 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:08.131056 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:08.131056 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:08.131056 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:08.131056 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:08.131056 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:08.131056 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:08.131056 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:08.131056 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:08.131056 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:14:08.131056 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:14:08.131056 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 87/188] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:14:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:14:08.131795 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82ac7e-d8b2-40c9-a5f0-e7cf306e0022 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:08.136201 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:08.157050 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 102/189] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:14:08.158695 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f371b-2895-4843-9741-7603eb5450d9 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:08.273211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82ac7e-d8b2-40c9-a5f0-e7cf306e0022 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:08.278458 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f371b-2895-4843-9741-7603eb5450d9 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:08.291889 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:08.316109 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7fd5f9-af49-49c4-be55-0dd0168e25a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:08.316664 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7fd5f9-af49-49c4-be55-0dd0168e25a6 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:08.316938 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7fd5f9-af49-49c4-be55-0dd0168e25a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:08.317059 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a5fa16-e21b-4870-9410-95d4d13abec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:08.317313 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7fd5f9-af49-49c4-be55-0dd0168e25a6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:08.317698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5fa16-e21b-4870-9410-95d4d13abec3 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:08.317941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5fa16-e21b-4870-9410-95d4d13abec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:08.318234 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5fa16-e21b-4870-9410-95d4d13abec3 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:08.319056 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a5fa16-e21b-4870-9410-95d4d13abec3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:08.319362 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a5fa16-e21b-4870-9410-95d4d13abec3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:08.321823 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a5fa16-e21b-4870-9410-95d4d13abec3 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:08.328855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7fd5f9-af49-49c4-be55-0dd0168e25a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:08.330275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 103/190] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:14:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:08.331205 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a5fa16-e21b-4870-9410-95d4d13abec3 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:08.337284 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-8d7fd5f9-af49-49c4-be55-0dd0168e25a6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:14:08.337284 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:08.337284 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:08.337284 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:08.337284 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:08.337284 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:08.337284 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:08.337284 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:08.337284 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:08.337284 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:08.337284 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:08.337284 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:08.337284 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:08.337284 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:08.337284 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:08.337284 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:08.338211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:08.338211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:08.338211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:08.338211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:08.338211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:08.338211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:08.338211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:08.338211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:08.338211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:08.338211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:08.338211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:08.338211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:08.338211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:08.338211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:08.338211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:08.338211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:08.339062 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:08.339062 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:08.339062 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:08.339062 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:08.339062 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:08.339062 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:08.339062 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:08.339062 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:08.339062 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:08.339062 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:08.339062 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:14:08.339062 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:14:08.339062 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 88/191] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:14:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:08.340316 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7fd5f9-af49-49c4-be55-0dd0168e25a6 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:08.395382 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:08.447201 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7fd5f9-af49-49c4-be55-0dd0168e25a6 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:08.448718 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a5fa16-e21b-4870-9410-95d4d13abec3 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:08.481235 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b57d736-3a72-48d6-8f13-3f8ae94a55b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:08.481890 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57d736-3a72-48d6-8f13-3f8ae94a55b3 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:08.482171 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57d736-3a72-48d6-8f13-3f8ae94a55b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:08.482445 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57d736-3a72-48d6-8f13-3f8ae94a55b3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:08.482537 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a1f109-196b-43b2-b855-29bdf3c1998b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:08.483134 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a1f109-196b-43b2-b855-29bdf3c1998b None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:08.483409 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a1f109-196b-43b2-b855-29bdf3c1998b None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:08.483683 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a1f109-196b-43b2-b855-29bdf3c1998b None admin] PATH_INFO: `/v3/users/ace8a130dc8a4a8485c9865314494b86` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:08.492263 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b57d736-3a72-48d6-8f13-3f8ae94a55b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:08.497449 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a1f109-196b-43b2-b855-29bdf3c1998b None admin] RBAC: Authorizing `identity:get_user(user_id=ace8a130dc8a4a8485c9865314494b86)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:08.498098 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-1b57d736-3a72-48d6-8f13-3f8ae94a55b3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:14:08.498098 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:08.498098 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:08.498098 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:08.498098 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:08.498098 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:08.498098 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:08.498098 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:08.498098 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:08.498098 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:08.498098 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:08.498098 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:08.498098 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:08.498098 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:08.498098 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:08.498098 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:08.499151 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:08.499151 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:08.499151 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:08.499151 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:08.499151 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:08.499151 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:08.499151 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:08.499151 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:08.499151 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:08.499151 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:08.499151 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:08.499151 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:08.499151 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:08.499151 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:08.499151 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:08.499151 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:08.502129 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:08.502129 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:08.502129 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:08.502129 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:08.502129 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:08.502129 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:08.502129 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:08.502129 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:08.502129 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:08.502129 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:08.502129 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:14:08.502129 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:08.502754 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a1f109-196b-43b2-b855-29bdf3c1998b None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:08.502840 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 104/192] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Sep 9 20:14:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:08.502840 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b57d736-3a72-48d6-8f13-3f8ae94a55b3 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:08.502959 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 89/193] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:14:08 2022] GET /identity/v3/users/ace8a130dc8a4a8485c9865314494b86 => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:08.502959 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a1f109-196b-43b2-b855-29bdf3c1998b None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:08.536206 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:08.605311 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a1f109-196b-43b2-b855-29bdf3c1998b None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:08.608980 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b57d736-3a72-48d6-8f13-3f8ae94a55b3 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:08.621896 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:08.639345 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5fae67-5257-450a-bff5-7ed9664f64e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:08.639963 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fae67-5257-450a-bff5-7ed9664f64e0 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:08.640239 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fae67-5257-450a-bff5-7ed9664f64e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:08.640508 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fae67-5257-450a-bff5-7ed9664f64e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:08.641617 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5fae67-5257-450a-bff5-7ed9664f64e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:08.643058 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5fae67-5257-450a-bff5-7ed9664f64e0 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:08.643864 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca88cc61-0c2c-4064-88ff-789a8a7fdf39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:08.644466 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca88cc61-0c2c-4064-88ff-789a8a7fdf39 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:08.644597 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 90/194] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:14:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:08.645369 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5fae67-5257-450a-bff5-7ed9664f64e0 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:08.645467 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca88cc61-0c2c-4064-88ff-789a8a7fdf39 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:08.645884 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca88cc61-0c2c-4064-88ff-789a8a7fdf39 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:08.646634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca88cc61-0c2c-4064-88ff-789a8a7fdf39 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:08.646949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca88cc61-0c2c-4064-88ff-789a8a7fdf39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:08.648558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca88cc61-0c2c-4064-88ff-789a8a7fdf39 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:08.657172 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 105/195] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Sep 9 20:14:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:08.658014 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca88cc61-0c2c-4064-88ff-789a8a7fdf39 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:08.742748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:08.746090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5fae67-5257-450a-bff5-7ed9664f64e0 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:08.758298 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca88cc61-0c2c-4064-88ff-789a8a7fdf39 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:08.779831 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec96595-9c41-40ec-b726-51ff2d4465ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:08.780544 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec96595-9c41-40ec-b726-51ff2d4465ee None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:08.780820 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec96595-9c41-40ec-b726-51ff2d4465ee None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:08.781097 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec96595-9c41-40ec-b726-51ff2d4465ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:08.794888 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc66c7-a41d-4cad-b6ce-cd520679e2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:08.795501 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc66c7-a41d-4cad-b6ce-cd520679e2e0 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:08.795771 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc66c7-a41d-4cad-b6ce-cd520679e2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:08.796042 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc66c7-a41d-4cad-b6ce-cd520679e2e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:08.797783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc66c7-a41d-4cad-b6ce-cd520679e2e0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:08.797783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc66c7-a41d-4cad-b6ce-cd520679e2e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:08.798560 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc66c7-a41d-4cad-b6ce-cd520679e2e0 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:08.807245 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 106/196] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:14:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:08.807245 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc66c7-a41d-4cad-b6ce-cd520679e2e0 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:08.841104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:08.856329 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec96595-9c41-40ec-b726-51ff2d4465ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:08.857782 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec96595-9c41-40ec-b726-51ff2d4465ee None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:08.908830 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc66c7-a41d-4cad-b6ce-cd520679e2e0 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:08.942822 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb24806-7cbf-470f-8b3f-2a1e7267fdaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:08.943427 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb24806-7cbf-470f-8b3f-2a1e7267fdaa None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:08.943706 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb24806-7cbf-470f-8b3f-2a1e7267fdaa None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:08.943979 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb24806-7cbf-470f-8b3f-2a1e7267fdaa None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:08.944624 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb24806-7cbf-470f-8b3f-2a1e7267fdaa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:08.946185 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb24806-7cbf-470f-8b3f-2a1e7267fdaa None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:08.954111 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:08.959309 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 91/197] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:14:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:14:08.960248 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec96595-9c41-40ec-b726-51ff2d4465ee None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:08.985272 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 107/198] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Sep 9 20:14:08 2022] POST /identity/v3/users => generated 297 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:14:08.985272 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb24806-7cbf-470f-8b3f-2a1e7267fdaa None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:09.081230 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec96595-9c41-40ec-b726-51ff2d4465ee None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:09.098515 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:09.113333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb24806-7cbf-470f-8b3f-2a1e7267fdaa None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:09.125403 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba035f3e-a975-44f8-9ec3-3a00758725d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:09.126443 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba035f3e-a975-44f8-9ec3-3a00758725d8 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:09.126873 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba035f3e-a975-44f8-9ec3-3a00758725d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:09.127394 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba035f3e-a975-44f8-9ec3-3a00758725d8 None admin] PATH_INFO: `/v3/users/13aeffc2d9264942acbb0e7d0e8ec830` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:09.128843 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba035f3e-a975-44f8-9ec3-3a00758725d8 None admin] RBAC: Authorizing `identity:get_user(user_id=13aeffc2d9264942acbb0e7d0e8ec830)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:09.130813 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba035f3e-a975-44f8-9ec3-3a00758725d8 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:09.132998 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 92/199] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:14:08 2022] GET /identity/v3/users/13aeffc2d9264942acbb0e7d0e8ec830 => generated 289 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:09.134082 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba035f3e-a975-44f8-9ec3-3a00758725d8 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:09.155482 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f209c4f-d1a5-49e8-b3cb-91c6ec30add2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:09.156130 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f209c4f-d1a5-49e8-b3cb-91c6ec30add2 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:09.156406 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f209c4f-d1a5-49e8-b3cb-91c6ec30add2 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:09.156673 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f209c4f-d1a5-49e8-b3cb-91c6ec30add2 None admin] PATH_INFO: `/v3/users/ace8a130dc8a4a8485c9865314494b86` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:09.158558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f209c4f-d1a5-49e8-b3cb-91c6ec30add2 None admin] RBAC: Authorizing `identity:get_user(user_id=ace8a130dc8a4a8485c9865314494b86)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:09.161111 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f209c4f-d1a5-49e8-b3cb-91c6ec30add2 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:09.162320 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 108/200] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:14:08 2022] GET /identity/v3/users/ace8a130dc8a4a8485c9865314494b86 => generated 283 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:09.163570 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f209c4f-d1a5-49e8-b3cb-91c6ec30add2 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:09.220554 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:09.265053 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba035f3e-a975-44f8-9ec3-3a00758725d8 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:09.302243 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f209c4f-d1a5-49e8-b3cb-91c6ec30add2 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:09.307798 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b9fea-c8c3-41d8-9066-c9e9f8e145aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:09.308446 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b9fea-c8c3-41d8-9066-c9e9f8e145aa None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:09.308727 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b9fea-c8c3-41d8-9066-c9e9f8e145aa None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:09.309001 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b9fea-c8c3-41d8-9066-c9e9f8e145aa None admin] PATH_INFO: `/v3/projects/445158de5da046e19c309424d250c953` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:09.310313 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b9fea-c8c3-41d8-9066-c9e9f8e145aa None admin] RBAC: Authorizing `identity:get_project(project_id=445158de5da046e19c309424d250c953)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:09.311786 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b9fea-c8c3-41d8-9066-c9e9f8e145aa None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:09.313700 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 93/201] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:14:09 2022] GET /identity/v3/projects/445158de5da046e19c309424d250c953 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:09.318445 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b9fea-c8c3-41d8-9066-c9e9f8e145aa None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:09.340850 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a7425-b9ff-485a-8700-576e763cdf93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:09.341551 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a7425-b9ff-485a-8700-576e763cdf93 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:09.341836 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a7425-b9ff-485a-8700-576e763cdf93 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:09.343039 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a7425-b9ff-485a-8700-576e763cdf93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:09.385058 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:09.437536 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b9fea-c8c3-41d8-9066-c9e9f8e145aa None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:09.438082 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4a7425-b9ff-485a-8700-576e763cdf93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:09.439691 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4a7425-b9ff-485a-8700-576e763cdf93 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:09.482773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015a76d-3056-4520-a15e-9a5eca46db73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:09.483390 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015a76d-3056-4520-a15e-9a5eca46db73 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:09.483674 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015a76d-3056-4520-a15e-9a5eca46db73 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:09.483941 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015a76d-3056-4520-a15e-9a5eca46db73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:09.488444 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:09.569290 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 109/202] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:14:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:14:09.569290 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a7425-b9ff-485a-8700-576e763cdf93 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:09.572950 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4015a76d-3056-4520-a15e-9a5eca46db73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:09.574622 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4015a76d-3056-4520-a15e-9a5eca46db73 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:09.630712 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:09.678238 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a7425-b9ff-485a-8700-576e763cdf93 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:09.700202 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 94/203] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:14:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:14:09.701290 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015a76d-3056-4520-a15e-9a5eca46db73 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:09.717376 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65ef9d-4f6d-4ab9-baec-4b8682d1b144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:09.718044 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65ef9d-4f6d-4ab9-baec-4b8682d1b144 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:09.718484 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65ef9d-4f6d-4ab9-baec-4b8682d1b144 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:09.718833 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65ef9d-4f6d-4ab9-baec-4b8682d1b144 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:09.719591 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d65ef9d-4f6d-4ab9-baec-4b8682d1b144 None admin] RBAC: Adding query filter params (role.id=9121ba25a6d84756990f0c3a19e9fecb, scope.project.id=445158de5da046e19c309424d250c953, user.id=ace8a130dc8a4a8485c9865314494b86) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:09.719999 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d65ef9d-4f6d-4ab9-baec-4b8682d1b144 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:09.721814 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d65ef9d-4f6d-4ab9-baec-4b8682d1b144 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:09.727501 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 110/204] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:14:09 2022] GET /identity/v3/role_assignments?user.id=ace8a130dc8a4a8485c9865314494b86&scope.project.id=445158de5da046e19c309424d250c953&role.id=9121ba25a6d84756990f0c3a19e9fecb => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:09.728383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65ef9d-4f6d-4ab9-baec-4b8682d1b144 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:09.759271 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:09.859153 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015a76d-3056-4520-a15e-9a5eca46db73 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:09.937393 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c8a9bd-1f0c-477f-9741-c090126b4b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:09.938045 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8a9bd-1f0c-477f-9741-c090126b4b4f None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:09.939413 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8a9bd-1f0c-477f-9741-c090126b4b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:09.939413 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8a9bd-1f0c-477f-9741-c090126b4b4f None admin] PATH_INFO: `/v3/projects/462d8ea458304464846aba18be1b8665` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:09.940352 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c8a9bd-1f0c-477f-9741-c090126b4b4f None admin] RBAC: Authorizing `identity:get_project(project_id=462d8ea458304464846aba18be1b8665)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:09.944091 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c8a9bd-1f0c-477f-9741-c090126b4b4f None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:09.944091 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 95/205] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:14:09 2022] GET /identity/v3/projects/462d8ea458304464846aba18be1b8665 => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:09.946478 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65ef9d-4f6d-4ab9-baec-4b8682d1b144 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:09.955226 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c8a9bd-1f0c-477f-9741-c090126b4b4f None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:09.998345 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:10.019406 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226b58d-1a2c-46f4-ac1a-f47e99828b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:10.020047 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226b58d-1a2c-46f4-ac1a-f47e99828b5c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:10.020326 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226b58d-1a2c-46f4-ac1a-f47e99828b5c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:10.020596 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226b58d-1a2c-46f4-ac1a-f47e99828b5c None admin] PATH_INFO: `/v3/projects/445158de5da046e19c309424d250c953` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:10.026564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a226b58d-1a2c-46f4-ac1a-f47e99828b5c None admin] RBAC: Authorizing `identity:get_project(project_id=445158de5da046e19c309424d250c953)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:10.028052 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a226b58d-1a2c-46f4-ac1a-f47e99828b5c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:10.029888 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 111/206] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:14:09 2022] GET /identity/v3/projects/445158de5da046e19c309424d250c953 => generated 302 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:10.036076 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226b58d-1a2c-46f4-ac1a-f47e99828b5c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:10.143356 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c8a9bd-1f0c-477f-9741-c090126b4b4f None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:10.145731 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:10.189678 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd9f0c-7561-4e38-9160-6c86806f4938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:10.190260 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd9f0c-7561-4e38-9160-6c86806f4938 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:10.190586 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd9f0c-7561-4e38-9160-6c86806f4938 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:10.190861 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd9f0c-7561-4e38-9160-6c86806f4938 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:10.191592 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efd9f0c-7561-4e38-9160-6c86806f4938 None admin] RBAC: Adding query filter params (role.id=3d90aea6ba6344ab9e478143d9be44fd, scope.project.id=462d8ea458304464846aba18be1b8665, user.id=13aeffc2d9264942acbb0e7d0e8ec830) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:10.191893 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efd9f0c-7561-4e38-9160-6c86806f4938 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:10.191992 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226b58d-1a2c-46f4-ac1a-f47e99828b5c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:10.197579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efd9f0c-7561-4e38-9160-6c86806f4938 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:10.209866 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 96/207] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:14:09 2022] GET /identity/v3/role_assignments?user.id=13aeffc2d9264942acbb0e7d0e8ec830&scope.project.id=462d8ea458304464846aba18be1b8665&role.id=3d90aea6ba6344ab9e478143d9be44fd => generated 608 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:10.231254 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef8467-ab9e-484f-935e-e220150a42b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:10.231874 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef8467-ab9e-484f-935e-e220150a42b5 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:10.232156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef8467-ab9e-484f-935e-e220150a42b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:10.232431 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef8467-ab9e-484f-935e-e220150a42b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:10.233175 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ef8467-ab9e-484f-935e-e220150a42b5 None admin] RBAC: Adding query filter params (role.id=d5528405b19b48a6bc4b7ff451332d8a, scope.project.id=445158de5da046e19c309424d250c953, user.id=ace8a130dc8a4a8485c9865314494b86) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:10.233509 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ef8467-ab9e-484f-935e-e220150a42b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:10.234938 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ef8467-ab9e-484f-935e-e220150a42b5 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:10.241370 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 112/208] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:14:10 2022] GET /identity/v3/role_assignments?user.id=ace8a130dc8a4a8485c9865314494b86&scope.project.id=445158de5da046e19c309424d250c953&role.id=d5528405b19b48a6bc4b7ff451332d8a => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:11.419370 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d92690-3a86-40fc-af95-33469d2a5f45 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:11.419938 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d92690-3a86-40fc-af95-33469d2a5f45 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:11.419938 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d92690-3a86-40fc-af95-33469d2a5f45 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:11.421337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 97/209] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:11.427541 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb3ed7-3c50-43c8-acae-51a042858fce None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:11.428563 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb3ed7-3c50-43c8-acae-51a042858fce None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:11.429042 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb3ed7-3c50-43c8-acae-51a042858fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:11.450307 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-6deb3ed7-3c50-43c8-acae-51a042858fce None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:11.451886 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6deb3ed7-3c50-43c8-acae-51a042858fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:11.509040 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:11.527527 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 113/210] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:11.561153 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01321078-2d97-45f1-8318-e9560a40e266 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:11.561519 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01321078-2d97-45f1-8318-e9560a40e266 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:11.561746 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01321078-2d97-45f1-8318-e9560a40e266 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:11.563095 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 98/211] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:11.568640 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234cf299-5d49-4fc3-b37c-0bc750a4a309 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:11.568928 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234cf299-5d49-4fc3-b37c-0bc750a4a309 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:11.569204 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234cf299-5d49-4fc3-b37c-0bc750a4a309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:11.588405 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-234cf299-5d49-4fc3-b37c-0bc750a4a309 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:11.589624 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-234cf299-5d49-4fc3-b37c-0bc750a4a309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:11.658829 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 114/212] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:11.800203 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01321078-2d97-45f1-8318-e9560a40e266 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:11.908031 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01321078-2d97-45f1-8318-e9560a40e266 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:11.937776 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234cf299-5d49-4fc3-b37c-0bc750a4a309 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:11.945441 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60768d39-80fd-4344-bb9a-04ecfda3d09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:11.946403 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60768d39-80fd-4344-bb9a-04ecfda3d09e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:11.946898 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60768d39-80fd-4344-bb9a-04ecfda3d09e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:11.947374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60768d39-80fd-4344-bb9a-04ecfda3d09e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:11.958885 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60768d39-80fd-4344-bb9a-04ecfda3d09e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:11.964773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-60768d39-80fd-4344-bb9a-04ecfda3d09e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:14:11.964773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:11.964773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:11.964773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:11.964773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:11.964773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:11.964773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:11.964773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:11.964773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:11.964773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:11.964773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:11.964773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:11.964773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:11.964773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:11.964773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:11.964773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:11.965781 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:11.965781 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:11.965781 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:11.965781 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:11.965781 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:11.965781 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:11.965781 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:11.965781 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:11.965781 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:11.965781 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:11.965781 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:11.965781 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:11.965781 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:11.965781 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:11.965781 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:11.965781 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:11.966570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:11.966570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:11.966570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:11.966570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:11.966570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:11.966570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:11.966570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:11.966570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:11.966570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:11.966570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:11.966570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:14:11.966570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:14:11.969521 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 99/213] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:14:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:14:11.977677 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60768d39-80fd-4344-bb9a-04ecfda3d09e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:12.016480 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:12.049774 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234cf299-5d49-4fc3-b37c-0bc750a4a309 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:12.067429 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:12.087395 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486b909-28db-4781-badd-70019d87e7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:12.088257 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486b909-28db-4781-badd-70019d87e7e9 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:12.088706 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486b909-28db-4781-badd-70019d87e7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:12.089117 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486b909-28db-4781-badd-70019d87e7e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:12.090404 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6486b909-28db-4781-badd-70019d87e7e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:12.091995 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6486b909-28db-4781-badd-70019d87e7e9 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:12.093798 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 115/214] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:14:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:12.099026 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4f10d-7021-406f-8ac1-86e22108a6be None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:12.099312 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4f10d-7021-406f-8ac1-86e22108a6be None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:12.099589 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4f10d-7021-406f-8ac1-86e22108a6be None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:12.100966 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 116/215] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:12.104269 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4f10d-7021-406f-8ac1-86e22108a6be None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:12.104406 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60768d39-80fd-4344-bb9a-04ecfda3d09e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:12.138251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04568cc7-311e-4b45-9790-4524af222c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:12.139120 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04568cc7-311e-4b45-9790-4524af222c65 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:12.139399 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04568cc7-311e-4b45-9790-4524af222c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:12.139670 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04568cc7-311e-4b45-9790-4524af222c65 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:12.140338 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04568cc7-311e-4b45-9790-4524af222c65 None admin] RBAC: Adding query filter params (name=member) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:12.140639 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04568cc7-311e-4b45-9790-4524af222c65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:12.142068 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04568cc7-311e-4b45-9790-4524af222c65 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:12.150938 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 100/216] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Sep 9 20:14:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:12.153392 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c531e-bff4-46ef-874a-69d2fae861d5 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:12.153392 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c531e-bff4-46ef-874a-69d2fae861d5 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:12.153392 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c531e-bff4-46ef-874a-69d2fae861d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:12.171752 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-cf6c531e-bff4-46ef-874a-69d2fae861d5 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:12.172935 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-cf6c531e-bff4-46ef-874a-69d2fae861d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:12.222283 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4f10d-7021-406f-8ac1-86e22108a6be None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:12.246724 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 101/217] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:12.256663 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef8c8a-8829-4bef-9ae9-cd6fa0e19fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:12.257296 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef8c8a-8829-4bef-9ae9-cd6fa0e19fa2 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:12.257579 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef8c8a-8829-4bef-9ae9-cd6fa0e19fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:12.257853 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef8c8a-8829-4bef-9ae9-cd6fa0e19fa2 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:12.258112 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c531e-bff4-46ef-874a-69d2fae861d5 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:12.258582 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef8c8a-8829-4bef-9ae9-cd6fa0e19fa2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:12.260017 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef8c8a-8829-4bef-9ae9-cd6fa0e19fa2 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:12.284406 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 117/218] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Sep 9 20:14:12 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:14:12.290760 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:12.381498 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c531e-bff4-46ef-874a-69d2fae861d5 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:12.420245 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c29eed-8c10-4bec-8e33-e780c4f45899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:12.421759 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c29eed-8c10-4bec-8e33-e780c4f45899 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:12.422322 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c29eed-8c10-4bec-8e33-e780c4f45899 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:12.422806 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c29eed-8c10-4bec-8e33-e780c4f45899 None admin] PATH_INFO: `/v3/users/24ea7c0437bd4f9f8c85fd85d1176576` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:12.439195 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c29eed-8c10-4bec-8e33-e780c4f45899 None admin] RBAC: Authorizing `identity:get_user(user_id=24ea7c0437bd4f9f8c85fd85d1176576)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:12.441004 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c29eed-8c10-4bec-8e33-e780c4f45899 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:12.443011 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 102/219] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:14:12 2022] GET /identity/v3/users/24ea7c0437bd4f9f8c85fd85d1176576 => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:12.449800 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef8c8a-8829-4bef-9ae9-cd6fa0e19fa2 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:12.500918 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:12.526697 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c29eed-8c10-4bec-8e33-e780c4f45899 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:12.583736 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef8c8a-8829-4bef-9ae9-cd6fa0e19fa2 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:12.628738 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:12.631870 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b2b1d6-4bc6-45d1-8066-5c82ade33eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:12.632504 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b2b1d6-4bc6-45d1-8066-5c82ade33eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:12.632785 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b2b1d6-4bc6-45d1-8066-5c82ade33eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:12.633063 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b2b1d6-4bc6-45d1-8066-5c82ade33eb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:12.663787 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c29eed-8c10-4bec-8e33-e780c4f45899 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:12.708508 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d3d75-afac-4c81-8411-80df56e83a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:12.709146 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d3d75-afac-4c81-8411-80df56e83a81 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:12.709456 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d3d75-afac-4c81-8411-80df56e83a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:12.709729 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d3d75-afac-4c81-8411-80df56e83a81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:12.723826 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776d3d75-afac-4c81-8411-80df56e83a81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:12.727260 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b2b1d6-4bc6-45d1-8066-5c82ade33eb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:12.728728 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b2b1d6-4bc6-45d1-8066-5c82ade33eb9 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:12.729002 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-776d3d75-afac-4c81-8411-80df56e83a81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:14:12.729002 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:12.729002 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:12.729002 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:12.729002 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:12.729002 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:12.729002 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:12.729002 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:12.729002 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:12.729002 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:12.729002 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:12.729002 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:12.729002 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:12.729002 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:12.729002 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:12.729002 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:12.729899 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:12.729899 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:12.729899 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:12.729899 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:12.729899 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:12.729899 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:12.729899 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:12.729899 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:12.729899 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:12.729899 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:12.729899 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:12.729899 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:12.729899 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:12.729899 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:12.729899 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:12.729899 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:12.730707 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:12.730707 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:12.730707 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:12.730707 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:12.730707 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:12.730707 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:12.730707 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:12.730707 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:12.730707 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:12.730707 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:12.730707 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:14:12.730707 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:14:12.730707 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 103/220] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:14:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:14:12.732762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50291f15-661f-4dd9-9cae-efc4241db331 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:12.733092 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50291f15-661f-4dd9-9cae-efc4241db331 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:12.733546 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50291f15-661f-4dd9-9cae-efc4241db331 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:12.735931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 104/221] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:12.741504 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50291f15-661f-4dd9-9cae-efc4241db331 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:12.787506 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:12.872030 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50291f15-661f-4dd9-9cae-efc4241db331 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:12.872497 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 118/222] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:14:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:14:12.875349 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a1a29-1bd7-40b7-a384-2450b74b404a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:12.875854 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a1a29-1bd7-40b7-a384-2450b74b404a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:12.876333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a1a29-1bd7-40b7-a384-2450b74b404a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:12.900047 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-173a1a29-1bd7-40b7-a384-2450b74b404a None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:12.901905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-173a1a29-1bd7-40b7-a384-2450b74b404a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:12.914393 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2c52e-4027-4303-ba32-536adb22f165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:12.915038 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2c52e-4027-4303-ba32-536adb22f165 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:12.915314 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2c52e-4027-4303-ba32-536adb22f165 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:12.915583 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2c52e-4027-4303-ba32-536adb22f165 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:12.916358 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c2c52e-4027-4303-ba32-536adb22f165 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:12.916559 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c2c52e-4027-4303-ba32-536adb22f165 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:12.917931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c2c52e-4027-4303-ba32-536adb22f165 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:12.924074 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:12.937731 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 105/223] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:14:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:12.938661 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2c52e-4027-4303-ba32-536adb22f165 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:12.993685 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 119/224] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:12.995413 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a1a29-1bd7-40b7-a384-2450b74b404a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:13.093725 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:13.113516 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a1a29-1bd7-40b7-a384-2450b74b404a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:13.154714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034503d-f7f7-43d5-b320-f3ed9c82001f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:13.156235 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034503d-f7f7-43d5-b320-f3ed9c82001f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:13.156748 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034503d-f7f7-43d5-b320-f3ed9c82001f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:13.157232 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034503d-f7f7-43d5-b320-f3ed9c82001f None admin] PATH_INFO: `/v3/users/ace8a130dc8a4a8485c9865314494b86` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:13.158594 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9034503d-f7f7-43d5-b320-f3ed9c82001f None admin] RBAC: Authorizing `identity:get_user(user_id=ace8a130dc8a4a8485c9865314494b86)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:13.160349 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9034503d-f7f7-43d5-b320-f3ed9c82001f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:13.162339 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 120/225] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:14:12 2022] GET /identity/v3/users/ace8a130dc8a4a8485c9865314494b86 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:13.168033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034503d-f7f7-43d5-b320-f3ed9c82001f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:13.187557 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2c52e-4027-4303-ba32-536adb22f165 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:13.285992 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4f782-ca9d-4a79-9160-9a3629e98a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:13.286655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4f782-ca9d-4a79-9160-9a3629e98a03 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:13.286933 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4f782-ca9d-4a79-9160-9a3629e98a03 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:13.287201 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4f782-ca9d-4a79-9160-9a3629e98a03 None admin] PATH_INFO: `/v3/projects/445158de5da046e19c309424d250c953` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:13.288373 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb4f782-ca9d-4a79-9160-9a3629e98a03 None admin] RBAC: Authorizing `identity:get_project(project_id=445158de5da046e19c309424d250c953)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:13.288887 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034503d-f7f7-43d5-b320-f3ed9c82001f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:13.289914 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb4f782-ca9d-4a79-9160-9a3629e98a03 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:13.292527 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 106/226] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:14:12 2022] GET /identity/v3/projects/445158de5da046e19c309424d250c953 => generated 302 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:13.297519 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4f782-ca9d-4a79-9160-9a3629e98a03 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:13.356836 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a653f0-7d74-48fe-a2c0-9d311fec2e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:13.357424 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a653f0-7d74-48fe-a2c0-9d311fec2e4a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:13.357707 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a653f0-7d74-48fe-a2c0-9d311fec2e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:13.357979 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a653f0-7d74-48fe-a2c0-9d311fec2e4a None admin] PATH_INFO: `/v3/projects/445158de5da046e19c309424d250c953` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:13.359244 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a653f0-7d74-48fe-a2c0-9d311fec2e4a None admin] RBAC: Authorizing `identity:get_project(project_id=445158de5da046e19c309424d250c953)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:13.360671 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a653f0-7d74-48fe-a2c0-9d311fec2e4a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:13.362552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 121/227] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:14:13 2022] GET /identity/v3/projects/445158de5da046e19c309424d250c953 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:13.372661 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a653f0-7d74-48fe-a2c0-9d311fec2e4a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:13.382725 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:13.408268 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:13.451418 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4f782-ca9d-4a79-9160-9a3629e98a03 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:13.499566 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beca67a-b256-4c34-9b9b-936183ad1b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:13.500284 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beca67a-b256-4c34-9b9b-936183ad1b3c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:13.500571 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beca67a-b256-4c34-9b9b-936183ad1b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:13.500951 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beca67a-b256-4c34-9b9b-936183ad1b3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:13.501786 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beca67a-b256-4c34-9b9b-936183ad1b3c None admin] RBAC: Adding query filter params (role.id=63fc659c11144d369a3f3fa684e056ae, scope.project.id=445158de5da046e19c309424d250c953, user.id=24ea7c0437bd4f9f8c85fd85d1176576) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:13.502139 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beca67a-b256-4c34-9b9b-936183ad1b3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:13.503689 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beca67a-b256-4c34-9b9b-936183ad1b3c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:13.507042 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a653f0-7d74-48fe-a2c0-9d311fec2e4a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:13.510568 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 107/228] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:14:13 2022] GET /identity/v3/role_assignments?user.id=24ea7c0437bd4f9f8c85fd85d1176576&scope.project.id=445158de5da046e19c309424d250c953&role.id=63fc659c11144d369a3f3fa684e056ae => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:13.511447 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beca67a-b256-4c34-9b9b-936183ad1b3c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:13.573465 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e216ca1-8e04-4f1a-9cab-82ccb4047a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:13.574148 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e216ca1-8e04-4f1a-9cab-82ccb4047a37 None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:13.574430 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e216ca1-8e04-4f1a-9cab-82ccb4047a37 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:13.574705 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e216ca1-8e04-4f1a-9cab-82ccb4047a37 None admin] PATH_INFO: `/v3/projects/445158de5da046e19c309424d250c953/users/ace8a130dc8a4a8485c9865314494b86/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:13.583045 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e216ca1-8e04-4f1a-9cab-82ccb4047a37 None admin] RBAC: Authorizing `identity:create_grant(project_id=445158de5da046e19c309424d250c953, user_id=ace8a130dc8a4a8485c9865314494b86, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:13.584513 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e216ca1-8e04-4f1a-9cab-82ccb4047a37 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:13.603731 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 122/229] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Sep 9 20:14:13 2022] PUT /identity/v3/projects/445158de5da046e19c309424d250c953/users/ace8a130dc8a4a8485c9865314494b86/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:14:13.628461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beca67a-b256-4c34-9b9b-936183ad1b3c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:13.665547 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:13.675572 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35457c4-9819-44d2-9440-bdd48abf9ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:13.676241 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35457c4-9819-44d2-9440-bdd48abf9ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:13.676525 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35457c4-9819-44d2-9440-bdd48abf9ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:13.676803 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35457c4-9819-44d2-9440-bdd48abf9ec1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:13.689652 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35457c4-9819-44d2-9440-bdd48abf9ec1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:13.695509 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-d35457c4-9819-44d2-9440-bdd48abf9ec1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:14:13.695509 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:13.695509 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:13.695509 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:13.695509 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:13.695509 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:13.695509 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:13.695509 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:13.695509 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:13.695509 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:13.695509 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:13.695509 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:13.695509 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:13.695509 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:13.695509 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:13.695509 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:13.697073 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:13.697073 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:13.697073 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:13.697073 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:13.697073 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:13.697073 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:13.697073 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:13.697073 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:13.697073 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:13.697073 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:13.697073 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:13.697073 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:13.697073 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:13.697073 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:13.697073 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:13.697073 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:13.697931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:13.697931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:13.697931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:13.697931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:13.697931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:13.697931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:13.697931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:13.697931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:13.697931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:13.697931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:13.697931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:14:13.697931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:14:13.697931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 108/230] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Sep 9 20:14:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:13.703735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e216ca1-8e04-4f1a-9cab-82ccb4047a37 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:13.797752 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:13.845387 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e216ca1-8e04-4f1a-9cab-82ccb4047a37 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:13.888637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5edfe4b-f071-490b-b7fb-341152620dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:13.889186 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5edfe4b-f071-490b-b7fb-341152620dee None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:13.889499 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5edfe4b-f071-490b-b7fb-341152620dee None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:13.889776 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5edfe4b-f071-490b-b7fb-341152620dee None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:13.890455 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5edfe4b-f071-490b-b7fb-341152620dee None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:13.890755 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5edfe4b-f071-490b-b7fb-341152620dee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:13.892127 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5edfe4b-f071-490b-b7fb-341152620dee None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:13.901785 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 123/231] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:14:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:13.907195 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35457c4-9819-44d2-9440-bdd48abf9ec1 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:14.017711 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35457c4-9819-44d2-9440-bdd48abf9ec1 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:14.063305 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16912605-5369-4179-8b95-5a703089e25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:14.063948 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16912605-5369-4179-8b95-5a703089e25b None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:14.064232 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16912605-5369-4179-8b95-5a703089e25b None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:14.064507 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16912605-5369-4179-8b95-5a703089e25b None admin] PATH_INFO: `/v3/users/ace8a130dc8a4a8485c9865314494b86` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:14.065680 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16912605-5369-4179-8b95-5a703089e25b None admin] RBAC: Authorizing `identity:get_user(user_id=ace8a130dc8a4a8485c9865314494b86)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:14.067124 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16912605-5369-4179-8b95-5a703089e25b None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:14.068699 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 109/232] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:14:13 2022] GET /identity/v3/users/ace8a130dc8a4a8485c9865314494b86 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:14.074965 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5edfe4b-f071-490b-b7fb-341152620dee None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:14.213502 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5edfe4b-f071-490b-b7fb-341152620dee None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:14.240268 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:14.248968 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f562c-bed8-459a-b230-84e90b04bc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:14.249658 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f562c-bed8-459a-b230-84e90b04bc6f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:14.250174 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f562c-bed8-459a-b230-84e90b04bc6f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:14.250791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f562c-bed8-459a-b230-84e90b04bc6f None admin] PATH_INFO: `/v3/projects/445158de5da046e19c309424d250c953` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:14.251938 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f562c-bed8-459a-b230-84e90b04bc6f None admin] RBAC: Authorizing `identity:get_project(project_id=445158de5da046e19c309424d250c953)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:14.253404 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f562c-bed8-459a-b230-84e90b04bc6f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:14.255633 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 124/233] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:14:14 2022] GET /identity/v3/projects/445158de5da046e19c309424d250c953 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:14.261816 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16912605-5369-4179-8b95-5a703089e25b None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:14.334736 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:14.376843 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16912605-5369-4179-8b95-5a703089e25b None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:14.412136 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f827a71f-5005-4f2c-b482-911d5ebb4fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:14.412841 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f827a71f-5005-4f2c-b482-911d5ebb4fea None admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:14.413133 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f827a71f-5005-4f2c-b482-911d5ebb4fea None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:14.413440 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f827a71f-5005-4f2c-b482-911d5ebb4fea None admin] PATH_INFO: `/v3/projects/445158de5da046e19c309424d250c953/users/ace8a130dc8a4a8485c9865314494b86/roles/d5528405b19b48a6bc4b7ff451332d8a` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:14.415671 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f827a71f-5005-4f2c-b482-911d5ebb4fea None admin] RBAC: Authorizing `identity:create_grant(project_id=445158de5da046e19c309424d250c953, user_id=ace8a130dc8a4a8485c9865314494b86, role_id=d5528405b19b48a6bc4b7ff451332d8a)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:14.417132 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f827a71f-5005-4f2c-b482-911d5ebb4fea None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:14.437759 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 110/234] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Sep 9 20:14:14 2022] PUT /identity/v3/projects/445158de5da046e19c309424d250c953/users/ace8a130dc8a4a8485c9865314494b86/roles/d5528405b19b48a6bc4b7ff451332d8a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:14:14.793897 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e8bb0-99d5-4492-872d-1efc609f333c None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:14.794412 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e8bb0-99d5-4492-872d-1efc609f333c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:14.794487 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e8bb0-99d5-4492-872d-1efc609f333c None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:14.795805 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 125/235] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:14.802711 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcec196-715d-4db7-977a-4347a9304cdc None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:14.803127 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcec196-715d-4db7-977a-4347a9304cdc None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:14.804662 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcec196-715d-4db7-977a-4347a9304cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:14.827861 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-dbcec196-715d-4db7-977a-4347a9304cdc None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:14.829405 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-dbcec196-715d-4db7-977a-4347a9304cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:14.932041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 111/236] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:15.040770 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8e9d3-f8b0-49da-8dbe-cebd4e3e55f4 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:15.041119 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8e9d3-f8b0-49da-8dbe-cebd4e3e55f4 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:15.041362 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8e9d3-f8b0-49da-8dbe-cebd4e3e55f4 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:15.043014 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 126/237] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:15.048212 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1e1da-4bad-47f4-99d9-b307d03763e1 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:15.048723 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1e1da-4bad-47f4-99d9-b307d03763e1 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:15.049208 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1e1da-4bad-47f4-99d9-b307d03763e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:15.072898 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-a0a1e1da-4bad-47f4-99d9-b307d03763e1 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:15.074453 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a0a1e1da-4bad-47f4-99d9-b307d03763e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:15.128646 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:15.159047 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 112/238] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:14:15.302781 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8e9d3-f8b0-49da-8dbe-cebd4e3e55f4 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:15.428891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8e9d3-f8b0-49da-8dbe-cebd4e3e55f4 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:15.472725 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e4d69-fc8e-4948-a37c-226151d16d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:15.473392 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e4d69-fc8e-4948-a37c-226151d16d14 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:15.473677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e4d69-fc8e-4948-a37c-226151d16d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:15.473949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e4d69-fc8e-4948-a37c-226151d16d14 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:15.487987 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e4d69-fc8e-4948-a37c-226151d16d14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:15.493514 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-9a9e4d69-fc8e-4948-a37c-226151d16d14 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:14:15.493514 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:15.493514 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:15.493514 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:15.493514 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:15.493514 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:15.493514 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:15.493514 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:15.493514 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:15.493514 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:15.493514 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:15.493514 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:15.493514 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:15.493514 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:15.493514 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:15.493514 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:15.494409 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:15.494409 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:15.494409 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:15.494409 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:15.494409 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:15.494409 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:15.494409 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:15.494409 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:15.494409 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:15.494409 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:15.494409 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:15.494409 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:15.494409 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:15.494409 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:15.494409 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:15.494409 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:15.495218 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:15.495218 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:15.495218 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:15.495218 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:15.495218 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:15.495218 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:15.495218 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:15.495218 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:15.495218 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:15.495218 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:15.495218 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:14:15.495218 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:15.495218 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 127/239] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:14:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:15.500652 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a1e1da-4bad-47f4-99d9-b307d03763e1 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:15.517932 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e4d69-fc8e-4948-a37c-226151d16d14 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:15.604664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:15.659898 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e4d69-fc8e-4948-a37c-226151d16d14 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:15.690414 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a1e1da-4bad-47f4-99d9-b307d03763e1 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:15.710607 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1db5be-65dc-45d5-8ac1-ce4a2935d020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:15.711251 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1db5be-65dc-45d5-8ac1-ce4a2935d020 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:15.711580 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1db5be-65dc-45d5-8ac1-ce4a2935d020 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:15.711860 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1db5be-65dc-45d5-8ac1-ce4a2935d020 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:15.715735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1db5be-65dc-45d5-8ac1-ce4a2935d020 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:15.717186 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1db5be-65dc-45d5-8ac1-ce4a2935d020 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:15.722707 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 128/240] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:14:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:15.728752 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1db5be-65dc-45d5-8ac1-ce4a2935d020 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:15.770767 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f3fac5-b2e9-47a2-ab00-1577fc157064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:15.771385 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3fac5-b2e9-47a2-ab00-1577fc157064 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:15.771666 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3fac5-b2e9-47a2-ab00-1577fc157064 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:15.771948 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3fac5-b2e9-47a2-ab00-1577fc157064 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:15.772625 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f3fac5-b2e9-47a2-ab00-1577fc157064 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:15.772927 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f3fac5-b2e9-47a2-ab00-1577fc157064 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:15.774883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f3fac5-b2e9-47a2-ab00-1577fc157064 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:15.796824 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 113/241] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Sep 9 20:14:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:15.797732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f3fac5-b2e9-47a2-ab00-1577fc157064 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:15.879241 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:15.887106 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1db5be-65dc-45d5-8ac1-ce4a2935d020 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:15.911414 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:15.928611 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb5692-b84c-4b1b-bfff-243edab56dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:15.929228 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb5692-b84c-4b1b-bfff-243edab56dca None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:15.929535 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb5692-b84c-4b1b-bfff-243edab56dca None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:15.929814 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb5692-b84c-4b1b-bfff-243edab56dca None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:15.931027 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbb5692-b84c-4b1b-bfff-243edab56dca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:15.932468 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbb5692-b84c-4b1b-bfff-243edab56dca None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:15.972121 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 129/242] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Sep 9 20:14:15 2022] POST /identity/v3/users => generated 293 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:14:16.000463 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f3fac5-b2e9-47a2-ab00-1577fc157064 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:16.044403 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baebd1cf-a6d5-43f8-92f3-6fa32cd9afdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:16.045053 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebd1cf-a6d5-43f8-92f3-6fa32cd9afdb None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:16.045385 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebd1cf-a6d5-43f8-92f3-6fa32cd9afdb None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:16.045677 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebd1cf-a6d5-43f8-92f3-6fa32cd9afdb None admin] PATH_INFO: `/v3/users/53a6edea2d7f4133b654f9b745feed07` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:16.067705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baebd1cf-a6d5-43f8-92f3-6fa32cd9afdb None admin] RBAC: Authorizing `identity:get_user(user_id=53a6edea2d7f4133b654f9b745feed07)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:16.069184 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baebd1cf-a6d5-43f8-92f3-6fa32cd9afdb None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:16.070980 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 114/243] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:14:15 2022] GET /identity/v3/users/53a6edea2d7f4133b654f9b745feed07 => generated 297 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:16.076724 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb5692-b84c-4b1b-bfff-243edab56dca None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:16.218136 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb5692-b84c-4b1b-bfff-243edab56dca None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:16.266381 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb5b8d-f1c4-437f-b1d1-4bc62522c281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:16.267018 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb5b8d-f1c4-437f-b1d1-4bc62522c281 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:16.267302 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb5b8d-f1c4-437f-b1d1-4bc62522c281 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:16.267577 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb5b8d-f1c4-437f-b1d1-4bc62522c281 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:16.303978 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bf364-24a8-4594-9e39-a35387ea5401 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:16.304287 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bf364-24a8-4594-9e39-a35387ea5401 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:16.304555 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bf364-24a8-4594-9e39-a35387ea5401 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:16.306702 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 115/244] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:14:16.312079 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf08357-2b09-4b59-be7b-d50d4ff8ca12 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:16.312355 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf08357-2b09-4b59-be7b-d50d4ff8ca12 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:16.312639 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf08357-2b09-4b59-be7b-d50d4ff8ca12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:16.337456 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-4bf08357-2b09-4b59-be7b-d50d4ff8ca12 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:16.338655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-4bf08357-2b09-4b59-be7b-d50d4ff8ca12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:16.362322 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:16.386327 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb5b8d-f1c4-437f-b1d1-4bc62522c281 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:16.388010 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb5b8d-f1c4-437f-b1d1-4bc62522c281 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:16.424787 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 116/245] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 09 20:14:16.499985 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 130/246] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:14:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:14:16.505490 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf08357-2b09-4b59-be7b-d50d4ff8ca12 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:16.548411 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:16.601452 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15d0f1-d4bd-4491-adf5-d316edfe69b2 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:16.602642 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15d0f1-d4bd-4491-adf5-d316edfe69b2 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:16.603124 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15d0f1-d4bd-4491-adf5-d316edfe69b2 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:16.604669 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 131/247] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:14:16.609949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15532812-6385-44d8-8854-a62b39c2afe0 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:16.610503 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15532812-6385-44d8-8854-a62b39c2afe0 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:16.610968 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15532812-6385-44d8-8854-a62b39c2afe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:16.616386 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf08357-2b09-4b59-be7b-d50d4ff8ca12 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:16.637442 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-15532812-6385-44d8-8854-a62b39c2afe0 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:16.639102 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-15532812-6385-44d8-8854-a62b39c2afe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:16.653054 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd3d01-54b0-406d-a2b9-e36c862ef2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:16.653728 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd3d01-54b0-406d-a2b9-e36c862ef2df None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:16.654090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd3d01-54b0-406d-a2b9-e36c862ef2df None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:16.654282 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd3d01-54b0-406d-a2b9-e36c862ef2df None admin] PATH_INFO: `/v3/projects/445158de5da046e19c309424d250c953` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:16.655845 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafd3d01-54b0-406d-a2b9-e36c862ef2df None admin] RBAC: Authorizing `identity:get_project(project_id=445158de5da046e19c309424d250c953)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:16.657513 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafd3d01-54b0-406d-a2b9-e36c862ef2df None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:16.661561 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 117/248] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:14:16 2022] GET /identity/v3/projects/445158de5da046e19c309424d250c953 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:16.668663 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd3d01-54b0-406d-a2b9-e36c862ef2df None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:16.744166 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 132/249] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:14:16.794942 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:16.800154 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd3d01-54b0-406d-a2b9-e36c862ef2df None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:16.810388 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15532812-6385-44d8-8854-a62b39c2afe0 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:16.824118 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:16.849218 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010b3c69-8afc-4ec3-97a6-a50f53a36ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:16.851312 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010b3c69-8afc-4ec3-97a6-a50f53a36ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:16.851312 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010b3c69-8afc-4ec3-97a6-a50f53a36ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:16.851516 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010b3c69-8afc-4ec3-97a6-a50f53a36ec6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:16.852233 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010b3c69-8afc-4ec3-97a6-a50f53a36ec6 None admin] RBAC: Adding query filter params (role.id=3d90aea6ba6344ab9e478143d9be44fd, scope.project.id=445158de5da046e19c309424d250c953, user.id=53a6edea2d7f4133b654f9b745feed07) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:16.852541 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010b3c69-8afc-4ec3-97a6-a50f53a36ec6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:16.855153 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010b3c69-8afc-4ec3-97a6-a50f53a36ec6 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:16.871832 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 118/250] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:14:16 2022] GET /identity/v3/role_assignments?user.id=53a6edea2d7f4133b654f9b745feed07&scope.project.id=445158de5da046e19c309424d250c953&role.id=3d90aea6ba6344ab9e478143d9be44fd => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:16.943924 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15532812-6385-44d8-8854-a62b39c2afe0 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:16.988537 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e9dc2-b18e-4438-b1cf-4cc0f020b19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:16.989573 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e9dc2-b18e-4438-b1cf-4cc0f020b19e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:16.990076 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e9dc2-b18e-4438-b1cf-4cc0f020b19e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:16.990582 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e9dc2-b18e-4438-b1cf-4cc0f020b19e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:17.007219 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e9dc2-b18e-4438-b1cf-4cc0f020b19e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:17.012461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-652e9dc2-b18e-4438-b1cf-4cc0f020b19e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:14:17.012461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:17.012461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:17.012461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:17.012461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:17.012461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:17.012461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:17.012461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:17.012461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:17.012461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:17.012461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:17.012461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:17.012461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:17.012461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:17.012461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:17.012461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:17.013383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:17.013383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:17.013383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:17.013383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:17.013383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:17.013383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:17.013383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:17.013383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:17.013383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:17.013383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:17.013383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:17.013383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:17.013383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:17.013383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:17.013383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:17.013383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:17.014186 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:17.014186 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:17.014186 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:17.014186 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:17.014186 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:17.014186 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:17.014186 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:17.014186 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:17.014186 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:17.014186 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:17.014186 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:14:17.014186 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:17.016742 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 133/251] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:14:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:14:17.023395 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010b3c69-8afc-4ec3-97a6-a50f53a36ec6 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:17.084440 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e9dc2-b18e-4438-b1cf-4cc0f020b19e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:17.226131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010b3c69-8afc-4ec3-97a6-a50f53a36ec6 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:17.249980 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:17.277466 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d6f3b-34d7-4442-aaa7-2d9af403d2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:17.278608 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d6f3b-34d7-4442-aaa7-2d9af403d2e2 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:17.279107 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d6f3b-34d7-4442-aaa7-2d9af403d2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:17.279580 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d6f3b-34d7-4442-aaa7-2d9af403d2e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:17.280454 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d6f3b-34d7-4442-aaa7-2d9af403d2e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:17.282008 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d6f3b-34d7-4442-aaa7-2d9af403d2e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:17.285115 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d6f3b-34d7-4442-aaa7-2d9af403d2e2 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:17.301694 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e9dc2-b18e-4438-b1cf-4cc0f020b19e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:17.305189 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 119/252] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Sep 9 20:14:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:17.307189 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78786621-4e6e-493d-9d4f-4fd61953d3d6 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:17.307475 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78786621-4e6e-493d-9d4f-4fd61953d3d6 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:17.307752 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78786621-4e6e-493d-9d4f-4fd61953d3d6 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:17.309976 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 120/253] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:17.311455 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78786621-4e6e-493d-9d4f-4fd61953d3d6 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:17.362459 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:17.376578 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e064bbfa-e5e9-4ed3-9545-15c380af24b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:17.377299 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064bbfa-e5e9-4ed3-9545-15c380af24b2 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:17.377583 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064bbfa-e5e9-4ed3-9545-15c380af24b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:17.377857 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064bbfa-e5e9-4ed3-9545-15c380af24b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:17.394176 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e064bbfa-e5e9-4ed3-9545-15c380af24b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:17.401640 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-e064bbfa-e5e9-4ed3-9545-15c380af24b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:14:17.401640 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:17.401640 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:17.401640 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:17.401640 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:17.401640 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:17.401640 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:17.401640 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:17.401640 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:17.401640 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:17.401640 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:17.401640 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:17.401640 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:17.401640 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:17.401640 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:17.401640 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:17.402592 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:17.402592 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:17.402592 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:17.402592 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:17.402592 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:17.402592 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:17.402592 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:17.402592 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:17.402592 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:17.402592 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:17.402592 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:17.402592 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:17.402592 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:17.402592 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:17.402592 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:17.402592 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:17.403860 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:17.403860 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:17.403860 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:17.403860 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:17.403860 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:17.403860 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:17.403860 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:17.403860 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:17.403860 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:17.403860 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:17.403860 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:14:17.403860 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:17.403860 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 134/254] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:14:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:14:17.404731 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd5ce2b-62f5-4e8f-a2a9-dedaf2a04486 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:17.404997 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd5ce2b-62f5-4e8f-a2a9-dedaf2a04486 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:17.405320 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd5ce2b-62f5-4e8f-a2a9-dedaf2a04486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:17.425391 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-edd5ce2b-62f5-4e8f-a2a9-dedaf2a04486 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:17.426666 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-edd5ce2b-62f5-4e8f-a2a9-dedaf2a04486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:17.511996 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78786621-4e6e-493d-9d4f-4fd61953d3d6 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:17.547915 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5348ce0c-2e98-47ea-a912-acef42bbd1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:17.548699 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348ce0c-2e98-47ea-a912-acef42bbd1e9 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:17.548981 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348ce0c-2e98-47ea-a912-acef42bbd1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:17.549339 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348ce0c-2e98-47ea-a912-acef42bbd1e9 None admin] PATH_INFO: `/v3/users/24ea7c0437bd4f9f8c85fd85d1176576` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:17.550457 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5348ce0c-2e98-47ea-a912-acef42bbd1e9 None admin] RBAC: Authorizing `identity:get_user(user_id=24ea7c0437bd4f9f8c85fd85d1176576)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:17.551890 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5348ce0c-2e98-47ea-a912-acef42bbd1e9 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:17.553838 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 121/255] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:14:17 2022] GET /identity/v3/users/24ea7c0437bd4f9f8c85fd85d1176576 => generated 297 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:17.554771 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5348ce0c-2e98-47ea-a912-acef42bbd1e9 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:17.562040 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 135/256] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:17.562872 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd5ce2b-62f5-4e8f-a2a9-dedaf2a04486 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:17.698694 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:17.699809 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:17.710283 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5348ce0c-2e98-47ea-a912-acef42bbd1e9 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:17.755069 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51b9d69-45db-42b8-b271-b471a83f8620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:17.756638 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b9d69-45db-42b8-b271-b471a83f8620 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:17.757182 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b9d69-45db-42b8-b271-b471a83f8620 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:17.758450 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b9d69-45db-42b8-b271-b471a83f8620 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:17.760343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51b9d69-45db-42b8-b271-b471a83f8620 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:17.760949 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51b9d69-45db-42b8-b271-b471a83f8620 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:17.763050 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51b9d69-45db-42b8-b271-b471a83f8620 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:17.772711 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd5ce2b-62f5-4e8f-a2a9-dedaf2a04486 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:17.780099 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 122/257] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:14:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:17.786446 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51b9d69-45db-42b8-b271-b471a83f8620 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:17.844217 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d9c1a5-418c-4cfc-93e1-8c5cd95bb44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:17.845094 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9c1a5-418c-4cfc-93e1-8c5cd95bb44d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:17.846370 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9c1a5-418c-4cfc-93e1-8c5cd95bb44d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:17.846881 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9c1a5-418c-4cfc-93e1-8c5cd95bb44d None admin] PATH_INFO: `/v3/projects/445158de5da046e19c309424d250c953` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:17.851153 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d9c1a5-418c-4cfc-93e1-8c5cd95bb44d None admin] RBAC: Authorizing `identity:get_project(project_id=445158de5da046e19c309424d250c953)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:17.853291 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d9c1a5-418c-4cfc-93e1-8c5cd95bb44d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:17.859862 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 136/258] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:14:17 2022] GET /identity/v3/projects/445158de5da046e19c309424d250c953 => generated 302 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:17.865022 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d9c1a5-418c-4cfc-93e1-8c5cd95bb44d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:17.939228 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51b9d69-45db-42b8-b271-b471a83f8620 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:17.975469 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca174fed-db5e-4d51-ae08-d410be93e94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:17.976498 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca174fed-db5e-4d51-ae08-d410be93e94d None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:17.977552 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca174fed-db5e-4d51-ae08-d410be93e94d None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:17.978017 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca174fed-db5e-4d51-ae08-d410be93e94d None admin] PATH_INFO: `/v3/users/ace8a130dc8a4a8485c9865314494b86` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:17.979424 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca174fed-db5e-4d51-ae08-d410be93e94d None admin] RBAC: Authorizing `identity:get_user(user_id=ace8a130dc8a4a8485c9865314494b86)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:17.981199 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca174fed-db5e-4d51-ae08-d410be93e94d None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:17.983446 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 123/259] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:14:17 2022] GET /identity/v3/users/ace8a130dc8a4a8485c9865314494b86 => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:17.988729 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca174fed-db5e-4d51-ae08-d410be93e94d None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:18.027510 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:18.049720 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d9c1a5-418c-4cfc-93e1-8c5cd95bb44d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:18.093613 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:18.109919 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111a97e-fe17-4245-9621-b7b5f4ea246e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:18.110601 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111a97e-fe17-4245-9621-b7b5f4ea246e None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:18.110872 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111a97e-fe17-4245-9621-b7b5f4ea246e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:18.111152 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111a97e-fe17-4245-9621-b7b5f4ea246e None admin] PATH_INFO: `/v3/projects/445158de5da046e19c309424d250c953/users/24ea7c0437bd4f9f8c85fd85d1176576/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:18.113139 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1111a97e-fe17-4245-9621-b7b5f4ea246e None admin] RBAC: Authorizing `identity:create_grant(project_id=445158de5da046e19c309424d250c953, user_id=24ea7c0437bd4f9f8c85fd85d1176576, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:18.114672 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1111a97e-fe17-4245-9621-b7b5f4ea246e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:18.116748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca174fed-db5e-4d51-ae08-d410be93e94d None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:18.147947 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 137/260] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Sep 9 20:14:17 2022] PUT /identity/v3/projects/445158de5da046e19c309424d250c953/users/24ea7c0437bd4f9f8c85fd85d1176576/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:14:18.157382 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111a97e-fe17-4245-9621-b7b5f4ea246e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:18.158232 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb6c49b-df53-4c3a-8bc1-aaac69f4bdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:18.159117 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6c49b-df53-4c3a-8bc1-aaac69f4bdda None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:18.159603 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6c49b-df53-4c3a-8bc1-aaac69f4bdda None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:18.160224 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6c49b-df53-4c3a-8bc1-aaac69f4bdda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:18.240592 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb6c49b-df53-4c3a-8bc1-aaac69f4bdda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:18.242531 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb6c49b-df53-4c3a-8bc1-aaac69f4bdda None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:18.270447 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:18.336666 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111a97e-fe17-4245-9621-b7b5f4ea246e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:18.363478 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 124/261] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:14:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 20:14:18.369543 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb6c49b-df53-4c3a-8bc1-aaac69f4bdda None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:18.407863 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e350f41-83c9-41fd-9f58-4ccd2110ca71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:18.408534 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e350f41-83c9-41fd-9f58-4ccd2110ca71 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:18.408818 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e350f41-83c9-41fd-9f58-4ccd2110ca71 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:18.409096 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e350f41-83c9-41fd-9f58-4ccd2110ca71 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:18.438427 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e350f41-83c9-41fd-9f58-4ccd2110ca71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:18.444285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-5e350f41-83c9-41fd-9f58-4ccd2110ca71 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:14:18.444285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:18.444285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:18.444285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:18.444285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:18.444285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:18.444285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:18.444285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:18.444285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:18.444285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:18.444285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:18.444285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:18.444285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:18.444285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:18.444285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:18.444285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:18.445273 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:18.445273 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:18.445273 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:18.445273 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:18.445273 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:18.445273 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:18.445273 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:18.445273 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:18.445273 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:18.445273 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:18.445273 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:18.445273 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:18.445273 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:18.445273 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:18.445273 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:18.445273 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:18.446145 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:18.446145 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:18.446145 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:18.446145 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:18.446145 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:18.446145 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:18.446145 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:18.446145 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:18.446145 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:18.446145 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:18.446145 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:14:18.446145 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:18.446145 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 138/262] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Sep 9 20:14:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:18.453534 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e350f41-83c9-41fd-9f58-4ccd2110ca71 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:18.505087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb6c49b-df53-4c3a-8bc1-aaac69f4bdda None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:18.529078 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:18.539048 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:18.545661 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63571d0c-9af3-434e-b9c2-4d0cc412fe08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:18.547503 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63571d0c-9af3-434e-b9c2-4d0cc412fe08 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:18.547976 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63571d0c-9af3-434e-b9c2-4d0cc412fe08 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:18.548493 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63571d0c-9af3-434e-b9c2-4d0cc412fe08 None admin] PATH_INFO: `/v3/projects/445158de5da046e19c309424d250c953` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:18.549863 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63571d0c-9af3-434e-b9c2-4d0cc412fe08 None admin] RBAC: Authorizing `identity:get_project(project_id=445158de5da046e19c309424d250c953)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:18.551627 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63571d0c-9af3-434e-b9c2-4d0cc412fe08 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:18.553471 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 125/263] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:14:18 2022] GET /identity/v3/projects/445158de5da046e19c309424d250c953 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:18.560407 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63571d0c-9af3-434e-b9c2-4d0cc412fe08 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:18.673807 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63571d0c-9af3-434e-b9c2-4d0cc412fe08 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:18.679420 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e350f41-83c9-41fd-9f58-4ccd2110ca71 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:18.709971 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3313b5a6-d4a7-4f00-91ba-fe2f8912f50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:18.710942 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313b5a6-d4a7-4f00-91ba-fe2f8912f50e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:18.711443 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313b5a6-d4a7-4f00-91ba-fe2f8912f50e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:18.711905 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313b5a6-d4a7-4f00-91ba-fe2f8912f50e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:18.712837 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3313b5a6-d4a7-4f00-91ba-fe2f8912f50e None admin] RBAC: Adding query filter params (role.id=9121ba25a6d84756990f0c3a19e9fecb, scope.project.id=445158de5da046e19c309424d250c953, user.id=ace8a130dc8a4a8485c9865314494b86) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:18.713377 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3313b5a6-d4a7-4f00-91ba-fe2f8912f50e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:18.715115 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3313b5a6-d4a7-4f00-91ba-fe2f8912f50e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:18.721307 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 126/264] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:14:18 2022] GET /identity/v3/role_assignments?user.id=ace8a130dc8a4a8485c9865314494b86&scope.project.id=445158de5da046e19c309424d250c953&role.id=9121ba25a6d84756990f0c3a19e9fecb => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:18.742139 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42fb4b8-a7dc-4532-aa5c-4fd4fad57e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:18.743714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42fb4b8-a7dc-4532-aa5c-4fd4fad57e0a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:18.744206 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42fb4b8-a7dc-4532-aa5c-4fd4fad57e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:18.744669 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42fb4b8-a7dc-4532-aa5c-4fd4fad57e0a None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:18.745559 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42fb4b8-a7dc-4532-aa5c-4fd4fad57e0a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:18.746052 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42fb4b8-a7dc-4532-aa5c-4fd4fad57e0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:18.747721 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42fb4b8-a7dc-4532-aa5c-4fd4fad57e0a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:18.757889 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 139/265] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:14:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:18.764115 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3313b5a6-d4a7-4f00-91ba-fe2f8912f50e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:18.825408 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:18.876359 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3313b5a6-d4a7-4f00-91ba-fe2f8912f50e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:18.913031 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36683c08-343d-4c6b-a5d2-694dd2dbe265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:18.913698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36683c08-343d-4c6b-a5d2-694dd2dbe265 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:18.913974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36683c08-343d-4c6b-a5d2-694dd2dbe265 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:18.914241 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36683c08-343d-4c6b-a5d2-694dd2dbe265 None admin] PATH_INFO: `/v3/users/ace8a130dc8a4a8485c9865314494b86` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:18.915982 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36683c08-343d-4c6b-a5d2-694dd2dbe265 None admin] RBAC: Authorizing `identity:get_user(user_id=ace8a130dc8a4a8485c9865314494b86)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:18.917495 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36683c08-343d-4c6b-a5d2-694dd2dbe265 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:18.919475 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 127/266] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:14:18 2022] GET /identity/v3/users/ace8a130dc8a4a8485c9865314494b86 => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:18.925197 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42fb4b8-a7dc-4532-aa5c-4fd4fad57e0a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:19.036214 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42fb4b8-a7dc-4532-aa5c-4fd4fad57e0a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:19.047908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:19.074578 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7268f1fa-c3cd-4be2-88c3-400912aceb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:19.075240 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268f1fa-c3cd-4be2-88c3-400912aceb31 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:19.075519 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268f1fa-c3cd-4be2-88c3-400912aceb31 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:19.075788 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268f1fa-c3cd-4be2-88c3-400912aceb31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:19.152693 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7268f1fa-c3cd-4be2-88c3-400912aceb31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:19.154258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7268f1fa-c3cd-4be2-88c3-400912aceb31 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:19.260351 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 140/267] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:14:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:14:19.266773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36683c08-343d-4c6b-a5d2-694dd2dbe265 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:19.388289 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36683c08-343d-4c6b-a5d2-694dd2dbe265 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:19.413278 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:19.425795 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8a25f-598e-4906-8c76-a459d550809a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:19.426486 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8a25f-598e-4906-8c76-a459d550809a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:19.426769 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8a25f-598e-4906-8c76-a459d550809a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:19.427040 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8a25f-598e-4906-8c76-a459d550809a None admin] PATH_INFO: `/v3/projects/445158de5da046e19c309424d250c953` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:19.428170 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda8a25f-598e-4906-8c76-a459d550809a None admin] RBAC: Authorizing `identity:get_project(project_id=445158de5da046e19c309424d250c953)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:19.429732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda8a25f-598e-4906-8c76-a459d550809a None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:19.432403 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 128/268] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:14:19 2022] GET /identity/v3/projects/445158de5da046e19c309424d250c953 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:19.438301 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7268f1fa-c3cd-4be2-88c3-400912aceb31 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:19.450622 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:19.550784 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7268f1fa-c3cd-4be2-88c3-400912aceb31 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:19.594692 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55f7dd-dc2b-4207-8900-c285d4e1132d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:19.595296 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55f7dd-dc2b-4207-8900-c285d4e1132d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:19.595579 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55f7dd-dc2b-4207-8900-c285d4e1132d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:19.595846 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55f7dd-dc2b-4207-8900-c285d4e1132d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:19.596573 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e55f7dd-dc2b-4207-8900-c285d4e1132d None admin] RBAC: Adding query filter params (role.id=d5528405b19b48a6bc4b7ff451332d8a, scope.project.id=445158de5da046e19c309424d250c953, user.id=ace8a130dc8a4a8485c9865314494b86) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:19.596875 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e55f7dd-dc2b-4207-8900-c285d4e1132d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:19.598325 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e55f7dd-dc2b-4207-8900-c285d4e1132d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:19.610581 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 141/269] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:14:19 2022] GET /identity/v3/role_assignments?user.id=ace8a130dc8a4a8485c9865314494b86&scope.project.id=445158de5da046e19c309424d250c953&role.id=d5528405b19b48a6bc4b7ff451332d8a => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:19.923489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e0a0a-855e-4e7e-b061-9b3678bd5052 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:19.923829 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e0a0a-855e-4e7e-b061-9b3678bd5052 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:19.923947 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e0a0a-855e-4e7e-b061-9b3678bd5052 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:19.925534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 129/270] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:19.932723 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3408d-8322-4c0f-bbf0-74002e033295 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:19.933010 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3408d-8322-4c0f-bbf0-74002e033295 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:19.933306 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3408d-8322-4c0f-bbf0-74002e033295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:19.947462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bb4be-8bae-4d91-bd89-9f4d7694e33f None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:19.947768 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bb4be-8bae-4d91-bd89-9f4d7694e33f None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:19.948007 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bb4be-8bae-4d91-bd89-9f4d7694e33f None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:19.949680 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 130/271] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:19.956826 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-6ae3408d-8322-4c0f-bbf0-74002e033295 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:19.958196 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6ae3408d-8322-4c0f-bbf0-74002e033295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:19.958454 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568800e-f6c8-4796-aabd-0e8f4923df0a None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:19.958735 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568800e-f6c8-4796-aabd-0e8f4923df0a None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:19.959002 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568800e-f6c8-4796-aabd-0e8f4923df0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:19.983404 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-e568800e-f6c8-4796-aabd-0e8f4923df0a None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:19.984587 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-e568800e-f6c8-4796-aabd-0e8f4923df0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:20.035815 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 142/272] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:14:20.061233 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 131/273] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:20.245749 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7ce2f-9756-4e58-82ee-4593475e2531 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:20.246262 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7ce2f-9756-4e58-82ee-4593475e2531 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:20.246347 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7ce2f-9756-4e58-82ee-4593475e2531 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:20.247912 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 143/274] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:20.252935 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1aa95-68a8-4b3c-8e8d-7916820def13 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:20.253507 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1aa95-68a8-4b3c-8e8d-7916820def13 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:20.253984 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1aa95-68a8-4b3c-8e8d-7916820def13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:20.274866 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-c2b1aa95-68a8-4b3c-8e8d-7916820def13 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:20.276815 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-c2b1aa95-68a8-4b3c-8e8d-7916820def13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:20.347460 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7ce2f-9756-4e58-82ee-4593475e2531 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:20.349841 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 132/275] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:20.353511 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:20.361560 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b1aa95-68a8-4b3c-8e8d-7916820def13 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:20.429604 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:20.471384 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7ce2f-9756-4e58-82ee-4593475e2531 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:20.496602 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b1aa95-68a8-4b3c-8e8d-7916820def13 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:20.507491 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85bb6f7-a0d6-4f75-be32-5c6458542de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:20.508702 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85bb6f7-a0d6-4f75-be32-5c6458542de1 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:20.508984 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85bb6f7-a0d6-4f75-be32-5c6458542de1 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:20.509278 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85bb6f7-a0d6-4f75-be32-5c6458542de1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:20.510392 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85bb6f7-a0d6-4f75-be32-5c6458542de1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:20.511829 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85bb6f7-a0d6-4f75-be32-5c6458542de1 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:20.513436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 144/276] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:14:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:20.518939 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85bb6f7-a0d6-4f75-be32-5c6458542de1 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:20.539441 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c6a6b-a494-448d-a834-b0bbf77022a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:20.540074 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c6a6b-a494-448d-a834-b0bbf77022a5 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:20.540350 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c6a6b-a494-448d-a834-b0bbf77022a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:20.540621 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c6a6b-a494-448d-a834-b0bbf77022a5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:20.556046 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618c6a6b-a494-448d-a834-b0bbf77022a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:20.562391 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-618c6a6b-a494-448d-a834-b0bbf77022a5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:14:20.562391 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:20.562391 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:20.562391 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:20.562391 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:20.562391 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:20.562391 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:20.562391 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:20.562391 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:20.562391 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:20.562391 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:20.562391 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:20.562391 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:20.562391 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:20.562391 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:20.562391 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:20.563286 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:20.563286 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:20.563286 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:20.563286 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:20.563286 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:20.563286 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:20.563286 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:20.563286 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:20.563286 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:20.563286 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:20.563286 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:20.563286 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:20.563286 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:20.563286 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:20.563286 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:20.563286 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:20.564204 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:20.564204 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:20.564204 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:20.564204 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:20.564204 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:20.564204 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:20.564204 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:20.564204 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:20.564204 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:20.564204 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:20.564204 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:14:20.564204 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:14:20.564816 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 133/277] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:14:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:14:20.572409 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c6a6b-a494-448d-a834-b0bbf77022a5 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:20.599401 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:20.638845 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85bb6f7-a0d6-4f75-be32-5c6458542de1 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:20.683391 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57dfcf-27cc-45d0-8f49-9152e33b3379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:20.684001 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57dfcf-27cc-45d0-8f49-9152e33b3379 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:20.684280 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57dfcf-27cc-45d0-8f49-9152e33b3379 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:20.684813 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57dfcf-27cc-45d0-8f49-9152e33b3379 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:20.685199 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e57dfcf-27cc-45d0-8f49-9152e33b3379 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:20.686650 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e57dfcf-27cc-45d0-8f49-9152e33b3379 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:20.696775 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c6a6b-a494-448d-a834-b0bbf77022a5 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:20.708588 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 145/278] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Sep 9 20:14:20 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:14:20.709175 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:20.715713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57dfcf-27cc-45d0-8f49-9152e33b3379 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:20.740011 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ffc2b4-a68b-4ace-807d-4c0418ca54e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:20.740741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ffc2b4-a68b-4ace-807d-4c0418ca54e6 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:20.741085 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ffc2b4-a68b-4ace-807d-4c0418ca54e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:20.741452 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ffc2b4-a68b-4ace-807d-4c0418ca54e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:20.742201 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ffc2b4-a68b-4ace-807d-4c0418ca54e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:20.743306 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ffc2b4-a68b-4ace-807d-4c0418ca54e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:20.744805 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ffc2b4-a68b-4ace-807d-4c0418ca54e6 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:20.756054 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 134/279] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Sep 9 20:14:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:20.762418 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ffc2b4-a68b-4ace-807d-4c0418ca54e6 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:20.877459 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57dfcf-27cc-45d0-8f49-9152e33b3379 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:20.891632 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ffc2b4-a68b-4ace-807d-4c0418ca54e6 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:20.922113 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:20.934320 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d653e-e7d8-4574-84a7-31054bc5265e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:20.934663 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d06fa8-ffe8-474d-ac75-4759d066f6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:20.935006 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d653e-e7d8-4574-84a7-31054bc5265e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:20.935282 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d653e-e7d8-4574-84a7-31054bc5265e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:20.935374 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d06fa8-ffe8-474d-ac75-4759d066f6f0 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:20.935550 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d653e-e7d8-4574-84a7-31054bc5265e None admin] PATH_INFO: `/v3/users/2882841c27404ce9a9fdbd1b5f010980` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:20.935666 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d06fa8-ffe8-474d-ac75-4759d066f6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:20.935844 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d06fa8-ffe8-474d-ac75-4759d066f6f0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:20.949282 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d06fa8-ffe8-474d-ac75-4759d066f6f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:20.952639 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d653e-e7d8-4574-84a7-31054bc5265e None admin] RBAC: Authorizing `identity:get_user(user_id=2882841c27404ce9a9fdbd1b5f010980)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:20.954370 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d653e-e7d8-4574-84a7-31054bc5265e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:20.954930 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-86d06fa8-ffe8-474d-ac75-4759d066f6f0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:14:20.954930 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:20.954930 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:20.954930 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:20.954930 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:20.954930 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:20.954930 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:20.954930 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:20.954930 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:20.954930 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:20.954930 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:20.954930 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:20.954930 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:20.954930 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:20.954930 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:20.954930 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:20.955795 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:20.955795 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:20.955795 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:20.955795 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:20.955795 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:20.955795 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:20.955795 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:20.955795 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:20.955795 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:20.955795 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:20.955795 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:20.955795 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:20.955795 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:20.955795 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:20.955795 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:20.955795 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:20.957585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:20.957585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:20.957585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:20.957585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:20.957585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:20.957585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:20.957585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:20.957585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:20.957585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:20.957585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:20.957585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:14:20.957585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:20.957585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 146/280] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:14:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:20.958243 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 135/281] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:14:20 2022] GET /identity/v3/users/2882841c27404ce9a9fdbd1b5f010980 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:20.961474 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d06fa8-ffe8-474d-ac75-4759d066f6f0 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:20.961986 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d653e-e7d8-4574-84a7-31054bc5265e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:20.997493 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:21.081903 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d06fa8-ffe8-474d-ac75-4759d066f6f0 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:21.083873 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d653e-e7d8-4574-84a7-31054bc5265e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:21.124156 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed8f880-78ad-4657-b001-520434cd8c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:21.124766 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8f880-78ad-4657-b001-520434cd8c6c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:21.125051 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8f880-78ad-4657-b001-520434cd8c6c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:21.126210 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8f880-78ad-4657-b001-520434cd8c6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:21.126210 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed8f880-78ad-4657-b001-520434cd8c6c None admin] RBAC: Adding query filter params (role.id=63fc659c11144d369a3f3fa684e056ae, user.id=2882841c27404ce9a9fdbd1b5f010980, scope.system=all) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:21.126457 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed8f880-78ad-4657-b001-520434cd8c6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:21.127829 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed8f880-78ad-4657-b001-520434cd8c6c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:21.128683 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a520c33b-e0c4-442d-a978-5aa927b8db73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:21.129584 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a520c33b-e0c4-442d-a978-5aa927b8db73 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:21.129871 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a520c33b-e0c4-442d-a978-5aa927b8db73 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:21.130143 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a520c33b-e0c4-442d-a978-5aa927b8db73 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:21.130906 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a520c33b-e0c4-442d-a978-5aa927b8db73 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:21.131207 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a520c33b-e0c4-442d-a978-5aa927b8db73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:21.132576 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a520c33b-e0c4-442d-a978-5aa927b8db73 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:21.134104 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 136/282] 10.222.0.26 () {56 vars in 1308 bytes} [Fri Sep 9 20:14:20 2022] GET /identity/v3/role_assignments?user.id=2882841c27404ce9a9fdbd1b5f010980&scope.system=all&role.id=63fc659c11144d369a3f3fa684e056ae => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:21.135485 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc01fc-6d82-48be-94cb-fc529e58d668 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:21.135949 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc01fc-6d82-48be-94cb-fc529e58d668 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:21.136411 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc01fc-6d82-48be-94cb-fc529e58d668 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:21.137762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 137/283] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:21.143367 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372f2e8-f997-45b7-b15f-2ad65346ed8e None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:21.143788 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372f2e8-f997-45b7-b15f-2ad65346ed8e None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:21.144187 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372f2e8-f997-45b7-b15f-2ad65346ed8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:21.144297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 147/284] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Sep 9 20:14:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:21.151615 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a520c33b-e0c4-442d-a978-5aa927b8db73 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:21.172103 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-5372f2e8-f997-45b7-b15f-2ad65346ed8e None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:21.173037 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-5372f2e8-f997-45b7-b15f-2ad65346ed8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:21.214547 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:21.259899 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 138/285] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:14:21.298943 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a520c33b-e0c4-442d-a978-5aa927b8db73 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:21.340864 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d21b2-57db-4fb6-b81e-13ef322b887c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:21.341777 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d21b2-57db-4fb6-b81e-13ef322b887c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:21.342203 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d21b2-57db-4fb6-b81e-13ef322b887c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:21.342624 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d21b2-57db-4fb6-b81e-13ef322b887c None admin] PATH_INFO: `/v3/users/53a6edea2d7f4133b654f9b745feed07` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:21.344029 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93d21b2-57db-4fb6-b81e-13ef322b887c None admin] RBAC: Authorizing `identity:get_user(user_id=53a6edea2d7f4133b654f9b745feed07)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:21.345744 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93d21b2-57db-4fb6-b81e-13ef322b887c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:21.347973 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 148/286] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:14:21 2022] GET /identity/v3/users/53a6edea2d7f4133b654f9b745feed07 => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:21.353937 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372f2e8-f997-45b7-b15f-2ad65346ed8e None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:21.385644 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:21.480868 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372f2e8-f997-45b7-b15f-2ad65346ed8e None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:21.521384 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52756d48-1fd1-455a-a6dc-d259cad17068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:21.522064 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52756d48-1fd1-455a-a6dc-d259cad17068 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:21.522346 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52756d48-1fd1-455a-a6dc-d259cad17068 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:21.522614 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52756d48-1fd1-455a-a6dc-d259cad17068 None admin] PATH_INFO: `/v3/projects/445158de5da046e19c309424d250c953` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:21.523756 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52756d48-1fd1-455a-a6dc-d259cad17068 None admin] RBAC: Authorizing `identity:get_project(project_id=445158de5da046e19c309424d250c953)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:21.525275 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52756d48-1fd1-455a-a6dc-d259cad17068 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:21.526925 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 139/287] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:14:21 2022] GET /identity/v3/projects/445158de5da046e19c309424d250c953 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:21.532972 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d21b2-57db-4fb6-b81e-13ef322b887c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:21.582816 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52756d48-1fd1-455a-a6dc-d259cad17068 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:21.711480 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d21b2-57db-4fb6-b81e-13ef322b887c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:21.734452 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:21.745131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:21.762851 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52756d48-1fd1-455a-a6dc-d259cad17068 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:21.768315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14347a-7594-4c2e-9fab-efbcd398f15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:21.768997 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14347a-7594-4c2e-9fab-efbcd398f15a None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:21.769293 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14347a-7594-4c2e-9fab-efbcd398f15a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:21.769573 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14347a-7594-4c2e-9fab-efbcd398f15a None admin] PATH_INFO: `/v3/projects/445158de5da046e19c309424d250c953/users/53a6edea2d7f4133b654f9b745feed07/roles/3d90aea6ba6344ab9e478143d9be44fd` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:21.771615 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db14347a-7594-4c2e-9fab-efbcd398f15a None admin] RBAC: Authorizing `identity:create_grant(project_id=445158de5da046e19c309424d250c953, user_id=53a6edea2d7f4133b654f9b745feed07, role_id=3d90aea6ba6344ab9e478143d9be44fd)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:21.773059 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db14347a-7594-4c2e-9fab-efbcd398f15a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:21.798538 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 149/288] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Sep 9 20:14:21 2022] PUT /identity/v3/projects/445158de5da046e19c309424d250c953/users/53a6edea2d7f4133b654f9b745feed07/roles/3d90aea6ba6344ab9e478143d9be44fd => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:14:21.829336 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596bab47-6d95-413f-ab8b-a59bdc906874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:21.830401 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596bab47-6d95-413f-ab8b-a59bdc906874 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:21.830959 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596bab47-6d95-413f-ab8b-a59bdc906874 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:21.831434 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596bab47-6d95-413f-ab8b-a59bdc906874 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:21.845064 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596bab47-6d95-413f-ab8b-a59bdc906874 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:21.850642 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-596bab47-6d95-413f-ab8b-a59bdc906874 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:14:21.850642 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:21.850642 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:21.850642 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:21.850642 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:21.850642 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:21.850642 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:21.850642 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:21.850642 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:21.850642 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:21.850642 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:21.850642 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:21.850642 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:21.850642 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:21.850642 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:21.850642 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:21.851608 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:21.851608 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:21.851608 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:21.851608 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:21.851608 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:21.851608 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:21.851608 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:21.851608 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:21.851608 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:21.851608 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:21.851608 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:21.851608 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:21.851608 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:21.851608 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:21.851608 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:21.851608 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:21.853028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:21.853028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:21.853028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:21.853028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:21.853028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:21.853028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:21.853028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:21.853028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:21.853028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:21.853028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:21.853028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:14:21.853028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:14:21.856006 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 140/289] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:14:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:14:21.861569 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14347a-7594-4c2e-9fab-efbcd398f15a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:22.014146 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14347a-7594-4c2e-9fab-efbcd398f15a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:22.055377 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc381e03-4efd-47b5-8cd0-1d656eacb5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:22.056004 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc381e03-4efd-47b5-8cd0-1d656eacb5f1 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:22.056283 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc381e03-4efd-47b5-8cd0-1d656eacb5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:22.056554 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc381e03-4efd-47b5-8cd0-1d656eacb5f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:22.057336 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc381e03-4efd-47b5-8cd0-1d656eacb5f1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:22.057644 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc381e03-4efd-47b5-8cd0-1d656eacb5f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:22.059278 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc381e03-4efd-47b5-8cd0-1d656eacb5f1 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:22.068077 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 150/290] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Sep 9 20:14:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:22.074588 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596bab47-6d95-413f-ab8b-a59bdc906874 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:22.217953 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596bab47-6d95-413f-ab8b-a59bdc906874 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:22.261974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ee95b2-e0b0-49c6-bec2-1fe889eb95b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:22.262610 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee95b2-e0b0-49c6-bec2-1fe889eb95b6 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:22.262888 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee95b2-e0b0-49c6-bec2-1fe889eb95b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:22.263159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee95b2-e0b0-49c6-bec2-1fe889eb95b6 None admin] PATH_INFO: `/v3/users/24ea7c0437bd4f9f8c85fd85d1176576` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:22.264347 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ee95b2-e0b0-49c6-bec2-1fe889eb95b6 None admin] RBAC: Authorizing `identity:get_user(user_id=24ea7c0437bd4f9f8c85fd85d1176576)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:22.265842 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ee95b2-e0b0-49c6-bec2-1fe889eb95b6 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:22.268704 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 141/291] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:14:22 2022] GET /identity/v3/users/24ea7c0437bd4f9f8c85fd85d1176576 => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:22.273343 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc381e03-4efd-47b5-8cd0-1d656eacb5f1 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:22.318961 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:22.395240 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc381e03-4efd-47b5-8cd0-1d656eacb5f1 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:22.447384 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2194611-9f7a-4088-a016-0e520a48ef47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:22.448006 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2194611-9f7a-4088-a016-0e520a48ef47 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:22.448286 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2194611-9f7a-4088-a016-0e520a48ef47 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:22.448559 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2194611-9f7a-4088-a016-0e520a48ef47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:22.541587 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2194611-9f7a-4088-a016-0e520a48ef47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:22.543352 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2194611-9f7a-4088-a016-0e520a48ef47 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:22.597609 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:22.659631 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 151/292] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:14:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:14:22.669264 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ee95b2-e0b0-49c6-bec2-1fe889eb95b6 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:22.709333 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:22.803271 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ee95b2-e0b0-49c6-bec2-1fe889eb95b6 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:22.839837 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce4fa8f-0dc3-4ce0-9de0-bfe34580ddec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:22.840524 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4fa8f-0dc3-4ce0-9de0-bfe34580ddec None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:22.840800 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4fa8f-0dc3-4ce0-9de0-bfe34580ddec None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:22.841120 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4fa8f-0dc3-4ce0-9de0-bfe34580ddec None admin] PATH_INFO: `/v3/projects/445158de5da046e19c309424d250c953` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:22.842279 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce4fa8f-0dc3-4ce0-9de0-bfe34580ddec None admin] RBAC: Authorizing `identity:get_project(project_id=445158de5da046e19c309424d250c953)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:22.843699 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce4fa8f-0dc3-4ce0-9de0-bfe34580ddec None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:22.845709 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 142/293] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:14:22 2022] GET /identity/v3/projects/445158de5da046e19c309424d250c953 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:22.851467 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2194611-9f7a-4088-a016-0e520a48ef47 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:23.005451 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2194611-9f7a-4088-a016-0e520a48ef47 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:23.049003 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24704cf6-8d77-428d-858f-311cb93218f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:23.049639 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24704cf6-8d77-428d-858f-311cb93218f4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:23.049912 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24704cf6-8d77-428d-858f-311cb93218f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:23.050179 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24704cf6-8d77-428d-858f-311cb93218f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:23.050909 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24704cf6-8d77-428d-858f-311cb93218f4 None admin] RBAC: Adding query filter params (role.id=63fc659c11144d369a3f3fa684e056ae, scope.project.id=445158de5da046e19c309424d250c953, user.id=24ea7c0437bd4f9f8c85fd85d1176576) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:23.051211 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24704cf6-8d77-428d-858f-311cb93218f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:23.052625 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24704cf6-8d77-428d-858f-311cb93218f4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:23.061980 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 152/294] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:14:22 2022] GET /identity/v3/role_assignments?user.id=24ea7c0437bd4f9f8c85fd85d1176576&scope.project.id=445158de5da046e19c309424d250c953&role.id=63fc659c11144d369a3f3fa684e056ae => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:23.485988 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94539b-1967-4486-a3ad-bfe7340e00e8 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:23.486217 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94539b-1967-4486-a3ad-bfe7340e00e8 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:23.486504 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94539b-1967-4486-a3ad-bfe7340e00e8 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:23.487873 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 143/295] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:23.493937 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa40c7-d22d-47d8-8f00-9ccce3bea2c6 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:23.494453 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa40c7-d22d-47d8-8f00-9ccce3bea2c6 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:23.494453 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa40c7-d22d-47d8-8f00-9ccce3bea2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:23.503728 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:23.518708 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-d4fa40c7-d22d-47d8-8f00-9ccce3bea2c6 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:23.519974 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d4fa40c7-d22d-47d8-8f00-9ccce3bea2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:23.594296 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 153/296] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:23.622130 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14f026-89f3-4c4c-963c-8441683ce1a0 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:23.622472 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14f026-89f3-4c4c-963c-8441683ce1a0 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:23.622674 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14f026-89f3-4c4c-963c-8441683ce1a0 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:23.624165 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 144/297] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:23.631443 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a78c38-1fb0-4f06-8298-e3915ef20848 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:23.631831 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a78c38-1fb0-4f06-8298-e3915ef20848 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:23.632200 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a78c38-1fb0-4f06-8298-e3915ef20848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:23.653280 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-17a78c38-1fb0-4f06-8298-e3915ef20848 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:23.654492 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-17a78c38-1fb0-4f06-8298-e3915ef20848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:23.742600 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 154/298] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:23.854715 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6119b90-5bc7-443b-943a-27a83804c4f3 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:23.855255 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6119b90-5bc7-443b-943a-27a83804c4f3 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:23.855255 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6119b90-5bc7-443b-943a-27a83804c4f3 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:23.856805 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 145/299] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:23.861908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0f0fe-faa6-42f1-b86c-db4ce7b4eb94 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:23.862253 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0f0fe-faa6-42f1-b86c-db4ce7b4eb94 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:23.862537 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0f0fe-faa6-42f1-b86c-db4ce7b4eb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:23.881580 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-dcb0f0fe-faa6-42f1-b86c-db4ce7b4eb94 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:23.883689 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-dcb0f0fe-faa6-42f1-b86c-db4ce7b4eb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:23.897368 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6119b90-5bc7-443b-943a-27a83804c4f3 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:23.957409 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:23.966945 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 155/300] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:23.993949 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:24.008431 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6119b90-5bc7-443b-943a-27a83804c4f3 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:24.045873 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f1f30-11e5-4729-b60a-4acaa928614a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:24.046714 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f1f30-11e5-4729-b60a-4acaa928614a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:24.047128 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f1f30-11e5-4729-b60a-4acaa928614a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:24.047558 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f1f30-11e5-4729-b60a-4acaa928614a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:24.048826 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905f1f30-11e5-4729-b60a-4acaa928614a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:24.050403 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905f1f30-11e5-4729-b60a-4acaa928614a None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:24.052128 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 146/301] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:14:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:24.058426 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0f0fe-faa6-42f1-b86c-db4ce7b4eb94 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:24.060724 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f1f30-11e5-4729-b60a-4acaa928614a None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:24.168099 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0f0fe-faa6-42f1-b86c-db4ce7b4eb94 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:24.171773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f1f30-11e5-4729-b60a-4acaa928614a None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:24.205331 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136cab31-193e-4220-925d-dfa6ebb25243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:24.205994 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cab31-193e-4220-925d-dfa6ebb25243 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:24.206383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cab31-193e-4220-925d-dfa6ebb25243 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:24.206708 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cab31-193e-4220-925d-dfa6ebb25243 None admin] PATH_INFO: `/v3/groups` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:24.207355 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136cab31-193e-4220-925d-dfa6ebb25243 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:24.208789 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136cab31-193e-4220-925d-dfa6ebb25243 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:24.211776 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f8133-35b0-4716-876d-0d94524e96c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:24.212902 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f8133-35b0-4716-876d-0d94524e96c5 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:24.213401 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f8133-35b0-4716-876d-0d94524e96c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:24.213873 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f8133-35b0-4716-876d-0d94524e96c5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:24.226258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 156/302] 10.222.0.26 () {60 vars in 1134 bytes} [Fri Sep 9 20:14:24 2022] POST /identity/v3/groups => generated 229 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:14:24.235333 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f8133-35b0-4716-876d-0d94524e96c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:24.241301 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-e92f8133-35b0-4716-876d-0d94524e96c5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:14:24.241301 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:24.241301 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:24.241301 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:24.241301 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:24.241301 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:24.241301 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:24.241301 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:24.241301 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:24.241301 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:24.241301 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:24.241301 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:24.241301 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:24.241301 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:24.241301 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:24.241301 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:24.243122 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:24.243122 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:24.243122 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:24.243122 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:24.243122 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:24.243122 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:24.243122 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:24.243122 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:24.243122 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:24.243122 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:24.243122 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:24.243122 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:24.243122 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:24.243122 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:24.243122 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:24.243122 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:24.244288 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:24.244288 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:24.244288 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:24.244288 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:24.244288 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:24.244288 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:24.244288 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:24.244288 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:24.244288 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:24.244288 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:24.244288 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:14:24.244288 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:14:24.244288 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 147/303] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:14:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:14:24.248048 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136cab31-193e-4220-925d-dfa6ebb25243 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:24.310549 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f8133-35b0-4716-876d-0d94524e96c5 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:24.329888 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:24.337322 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:24.396635 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136cab31-193e-4220-925d-dfa6ebb25243 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:24.435063 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb72ecd-0033-4da1-8b35-0793459c1a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:24.435683 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb72ecd-0033-4da1-8b35-0793459c1a3b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:24.435961 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb72ecd-0033-4da1-8b35-0793459c1a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:24.436230 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb72ecd-0033-4da1-8b35-0793459c1a3b None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:24.438211 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb72ecd-0033-4da1-8b35-0793459c1a3b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:24.438211 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb72ecd-0033-4da1-8b35-0793459c1a3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:24.438584 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb72ecd-0033-4da1-8b35-0793459c1a3b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:24.447984 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 157/304] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Sep 9 20:14:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:24.453908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb72ecd-0033-4da1-8b35-0793459c1a3b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:24.460640 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f8133-35b0-4716-876d-0d94524e96c5 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:24.504002 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12b328-989f-4d9d-a458-9f70378b030a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:24.504987 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12b328-989f-4d9d-a458-9f70378b030a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:24.505352 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12b328-989f-4d9d-a458-9f70378b030a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:24.505694 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12b328-989f-4d9d-a458-9f70378b030a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:24.519731 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af12b328-989f-4d9d-a458-9f70378b030a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:24.526459 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-af12b328-989f-4d9d-a458-9f70378b030a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:14:24.526459 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:24.526459 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:24.526459 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:24.526459 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:24.526459 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:24.526459 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:24.526459 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:24.526459 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:24.526459 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:24.526459 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:24.526459 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:24.526459 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:24.526459 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:24.526459 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:24.526459 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:24.527418 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:24.527418 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:24.527418 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:24.527418 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:24.527418 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:24.527418 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:24.527418 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:24.527418 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:24.527418 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:24.527418 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:24.527418 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:24.527418 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:24.527418 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:24.527418 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:24.527418 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:24.527418 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:24.528210 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:24.528210 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:24.528210 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:24.528210 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:24.528210 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:24.528210 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:24.528210 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:24.528210 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:24.528210 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:24.528210 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:24.528210 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:14:24.528210 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:14:24.528210 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 148/305] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:14:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:24.534174 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12b328-989f-4d9d-a458-9f70378b030a None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:24.578427 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb72ecd-0033-4da1-8b35-0793459c1a3b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:24.601010 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:24.624141 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0015931-6f30-4916-ae41-99f9d3373446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:24.624790 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0015931-6f30-4916-ae41-99f9d3373446 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:24.625074 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0015931-6f30-4916-ae41-99f9d3373446 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:24.625381 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0015931-6f30-4916-ae41-99f9d3373446 None admin] PATH_INFO: `/v3/users/bc399701980047c39d834b377a0233e4` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:24.640865 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:24.649444 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0015931-6f30-4916-ae41-99f9d3373446 None admin] RBAC: Authorizing `identity:get_user(user_id=bc399701980047c39d834b377a0233e4)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:24.651451 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0015931-6f30-4916-ae41-99f9d3373446 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:24.653880 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 158/306] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:14:24 2022] GET /identity/v3/users/bc399701980047c39d834b377a0233e4 => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:24.660087 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0015931-6f30-4916-ae41-99f9d3373446 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:24.679425 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12b328-989f-4d9d-a458-9f70378b030a None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:24.716961 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c63cb-ddd8-4d66-b379-79d554d16131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:24.717808 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c63cb-ddd8-4d66-b379-79d554d16131 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:24.718433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c63cb-ddd8-4d66-b379-79d554d16131 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:24.718433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c63cb-ddd8-4d66-b379-79d554d16131 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:24.719274 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179c63cb-ddd8-4d66-b379-79d554d16131 None admin] RBAC: Adding query filter params (name=member) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:24.719671 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179c63cb-ddd8-4d66-b379-79d554d16131 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:24.721140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179c63cb-ddd8-4d66-b379-79d554d16131 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:24.734638 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 149/307] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Sep 9 20:14:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:24.736351 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c63cb-ddd8-4d66-b379-79d554d16131 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:24.794541 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0015931-6f30-4916-ae41-99f9d3373446 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:24.832432 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc9bc4b-a156-4644-876b-f3a40bf1ca12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:24.833038 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9bc4b-a156-4644-876b-f3a40bf1ca12 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:24.833337 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9bc4b-a156-4644-876b-f3a40bf1ca12 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:24.833609 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9bc4b-a156-4644-876b-f3a40bf1ca12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:24.834328 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc9bc4b-a156-4644-876b-f3a40bf1ca12 None admin] RBAC: Adding query filter params (role.id=3d90aea6ba6344ab9e478143d9be44fd, user.id=bc399701980047c39d834b377a0233e4, scope.system=all) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:24.834631 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc9bc4b-a156-4644-876b-f3a40bf1ca12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:24.836084 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc9bc4b-a156-4644-876b-f3a40bf1ca12 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:24.840989 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 159/308] 10.222.0.26 () {56 vars in 1308 bytes} [Fri Sep 9 20:14:24 2022] GET /identity/v3/role_assignments?user.id=bc399701980047c39d834b377a0233e4&scope.system=all&role.id=3d90aea6ba6344ab9e478143d9be44fd => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:24.842176 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddbb8c4-182b-4042-b4af-62c9de52e7ce None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:24.842454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddbb8c4-182b-4042-b4af-62c9de52e7ce None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:24.842724 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddbb8c4-182b-4042-b4af-62c9de52e7ce None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:24.844083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 160/309] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:24.847046 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c63cb-ddd8-4d66-b379-79d554d16131 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:24.850527 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e363b-e7c5-4be8-96ef-1fe7b6aa696a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:24.850527 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e363b-e7c5-4be8-96ef-1fe7b6aa696a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:24.850527 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e363b-e7c5-4be8-96ef-1fe7b6aa696a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:24.870229 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-1d3e363b-e7c5-4be8-96ef-1fe7b6aa696a None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:24.871459 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1d3e363b-e7c5-4be8-96ef-1fe7b6aa696a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:24.892211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce4a99-f131-4ca2-89ec-5bb027665e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:24.893646 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce4a99-f131-4ca2-89ec-5bb027665e5f None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:24.894130 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce4a99-f131-4ca2-89ec-5bb027665e5f None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:24.894631 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce4a99-f131-4ca2-89ec-5bb027665e5f None admin] PATH_INFO: `/v3/users/2882841c27404ce9a9fdbd1b5f010980` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:24.896223 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce4a99-f131-4ca2-89ec-5bb027665e5f None admin] RBAC: Authorizing `identity:get_user(user_id=2882841c27404ce9a9fdbd1b5f010980)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:24.897988 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce4a99-f131-4ca2-89ec-5bb027665e5f None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:24.901927 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 150/310] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:14:24 2022] GET /identity/v3/users/2882841c27404ce9a9fdbd1b5f010980 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:24.908944 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce4a99-f131-4ca2-89ec-5bb027665e5f None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:24.916079 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:24.921129 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:24.970506 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 161/311] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:14:25.043480 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce4a99-f131-4ca2-89ec-5bb027665e5f None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:25.078418 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba050d-3215-44a2-9547-238058e4068e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:25.078989 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba050d-3215-44a2-9547-238058e4068e None admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:25.079265 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba050d-3215-44a2-9547-238058e4068e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:25.079533 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba050d-3215-44a2-9547-238058e4068e None admin] PATH_INFO: `/v3/system/users/2882841c27404ce9a9fdbd1b5f010980/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:25.080920 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba050d-3215-44a2-9547-238058e4068e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2882841c27404ce9a9fdbd1b5f010980, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:25.082364 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba050d-3215-44a2-9547-238058e4068e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:25.092160 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 151/312] 10.222.0.26 () {58 vars in 1334 bytes} [Fri Sep 9 20:14:24 2022] PUT /identity/v3/system/users/2882841c27404ce9a9fdbd1b5f010980/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:14:25.260230 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e363b-e7c5-4be8-96ef-1fe7b6aa696a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:25.373978 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e363b-e7c5-4be8-96ef-1fe7b6aa696a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:25.409525 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0217972f-937e-4db3-a36e-97b61f416aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:25.410164 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217972f-937e-4db3-a36e-97b61f416aa0 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:25.410446 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217972f-937e-4db3-a36e-97b61f416aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:25.410722 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217972f-937e-4db3-a36e-97b61f416aa0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:25.420916 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0217972f-937e-4db3-a36e-97b61f416aa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:25.425908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-0217972f-937e-4db3-a36e-97b61f416aa0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:14:25.425908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:25.425908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:25.425908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:25.425908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:25.425908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:25.425908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:25.425908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:25.425908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:25.425908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:25.425908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:25.425908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:25.425908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:25.425908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:25.425908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:25.425908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:25.426790 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:25.426790 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:25.426790 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:25.426790 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:25.426790 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:25.426790 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:25.426790 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:25.426790 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:25.426790 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:25.426790 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:25.426790 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:25.426790 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:25.426790 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:25.426790 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:25.426790 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:25.426790 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:25.427608 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:25.427608 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:25.427608 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:25.427608 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:25.427608 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:25.427608 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:25.427608 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:25.427608 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:25.427608 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:25.427608 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:25.427608 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:14:25.427608 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:25.427608 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 162/313] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:14:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:25.434515 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba050d-3215-44a2-9547-238058e4068e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:25.504703 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:25.538547 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba050d-3215-44a2-9547-238058e4068e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:25.576647 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a377f9d2-0db9-46b3-879c-6a385a344001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:25.577276 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377f9d2-0db9-46b3-879c-6a385a344001 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:25.577552 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377f9d2-0db9-46b3-879c-6a385a344001 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:25.577823 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377f9d2-0db9-46b3-879c-6a385a344001 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:25.578484 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a377f9d2-0db9-46b3-879c-6a385a344001 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:25.578779 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a377f9d2-0db9-46b3-879c-6a385a344001 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:25.580149 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a377f9d2-0db9-46b3-879c-6a385a344001 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:25.589001 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 152/314] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Sep 9 20:14:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:25.594505 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0217972f-937e-4db3-a36e-97b61f416aa0 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:25.624570 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:25.705816 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0217972f-937e-4db3-a36e-97b61f416aa0 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:25.743384 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e668ba-df20-49b9-a419-ee3bf9d14444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:25.744022 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e668ba-df20-49b9-a419-ee3bf9d14444 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:25.744298 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e668ba-df20-49b9-a419-ee3bf9d14444 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:25.744572 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e668ba-df20-49b9-a419-ee3bf9d14444 None admin] PATH_INFO: `/v3/users/53a6edea2d7f4133b654f9b745feed07` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:25.745745 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e668ba-df20-49b9-a419-ee3bf9d14444 None admin] RBAC: Authorizing `identity:get_user(user_id=53a6edea2d7f4133b654f9b745feed07)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:25.747166 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e668ba-df20-49b9-a419-ee3bf9d14444 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:25.748741 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 163/315] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:14:25 2022] GET /identity/v3/users/53a6edea2d7f4133b654f9b745feed07 => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:25.754489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a377f9d2-0db9-46b3-879c-6a385a344001 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:25.862396 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a377f9d2-0db9-46b3-879c-6a385a344001 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:25.892339 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:25.897309 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e675ae83-33e8-41c4-a528-35d5ea8385a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:25.898247 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e675ae83-33e8-41c4-a528-35d5ea8385a6 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:25.898767 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e675ae83-33e8-41c4-a528-35d5ea8385a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:25.899254 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e675ae83-33e8-41c4-a528-35d5ea8385a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:25.973665 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e675ae83-33e8-41c4-a528-35d5ea8385a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:25.975466 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e675ae83-33e8-41c4-a528-35d5ea8385a6 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:26.079707 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 153/316] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:14:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:14:26.086106 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e668ba-df20-49b9-a419-ee3bf9d14444 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:26.183197 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:26.189793 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e668ba-df20-49b9-a419-ee3bf9d14444 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:26.224206 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb8b21-a04d-4431-a44f-da187289cdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:26.224837 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb8b21-a04d-4431-a44f-da187289cdd4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:26.225115 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb8b21-a04d-4431-a44f-da187289cdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:26.225411 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb8b21-a04d-4431-a44f-da187289cdd4 None admin] PATH_INFO: `/v3/projects/445158de5da046e19c309424d250c953` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:26.227030 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb8b21-a04d-4431-a44f-da187289cdd4 None admin] RBAC: Authorizing `identity:get_project(project_id=445158de5da046e19c309424d250c953)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:26.228469 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb8b21-a04d-4431-a44f-da187289cdd4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:26.230113 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 164/317] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:14:26 2022] GET /identity/v3/projects/445158de5da046e19c309424d250c953 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:26.235950 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e675ae83-33e8-41c4-a528-35d5ea8385a6 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:26.267343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:26.338271 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e675ae83-33e8-41c4-a528-35d5ea8385a6 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:26.372805 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91561c9-8f52-4c44-8c20-6431d19737d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:26.373462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91561c9-8f52-4c44-8c20-6431d19737d2 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:26.373736 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91561c9-8f52-4c44-8c20-6431d19737d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:26.374005 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91561c9-8f52-4c44-8c20-6431d19737d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:26.374726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91561c9-8f52-4c44-8c20-6431d19737d2 None admin] RBAC: Adding query filter params (role.id=3d90aea6ba6344ab9e478143d9be44fd, scope.project.id=445158de5da046e19c309424d250c953, user.id=53a6edea2d7f4133b654f9b745feed07) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:26.375041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91561c9-8f52-4c44-8c20-6431d19737d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:26.376559 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91561c9-8f52-4c44-8c20-6431d19737d2 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:26.382953 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 154/318] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:14:26 2022] GET /identity/v3/role_assignments?user.id=53a6edea2d7f4133b654f9b745feed07&scope.project.id=445158de5da046e19c309424d250c953&role.id=3d90aea6ba6344ab9e478143d9be44fd => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:26.566538 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99521392-77aa-4af7-84b4-84a8a6905521 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:26.566826 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99521392-77aa-4af7-84b4-84a8a6905521 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:26.567101 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99521392-77aa-4af7-84b4-84a8a6905521 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:26.568450 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 165/319] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:26.573807 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c57407-8fa7-4018-8c56-90275cbdcae6 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:26.574091 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c57407-8fa7-4018-8c56-90275cbdcae6 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:26.574368 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c57407-8fa7-4018-8c56-90275cbdcae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:26.594019 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-19c57407-8fa7-4018-8c56-90275cbdcae6 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:26.595246 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-19c57407-8fa7-4018-8c56-90275cbdcae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:26.667728 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 155/320] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:26.935413 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99521392-77aa-4af7-84b4-84a8a6905521 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:27.042987 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99521392-77aa-4af7-84b4-84a8a6905521 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:27.081127 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c11c9ae-0a79-46d7-8f51-6dd4de703f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:27.081786 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c11c9ae-0a79-46d7-8f51-6dd4de703f37 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:27.082060 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c11c9ae-0a79-46d7-8f51-6dd4de703f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:27.082329 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c11c9ae-0a79-46d7-8f51-6dd4de703f37 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:27.083422 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c11c9ae-0a79-46d7-8f51-6dd4de703f37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:27.084832 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c11c9ae-0a79-46d7-8f51-6dd4de703f37 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:27.086665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 166/321] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:14:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:27.092098 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c57407-8fa7-4018-8c56-90275cbdcae6 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:27.118712 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:27.159424 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c3c69-fb13-4d67-8b68-3e23cdb5b998 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:27.159714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c3c69-fb13-4d67-8b68-3e23cdb5b998 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:27.159990 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c3c69-fb13-4d67-8b68-3e23cdb5b998 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:27.161483 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 167/322] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:14:27.166166 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f55a05a-1658-4bb0-a50d-34bcf716c2b6 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:27.166450 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f55a05a-1658-4bb0-a50d-34bcf716c2b6 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:27.166722 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f55a05a-1658-4bb0-a50d-34bcf716c2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:27.186132 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-8f55a05a-1658-4bb0-a50d-34bcf716c2b6 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:27.187250 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8f55a05a-1658-4bb0-a50d-34bcf716c2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:27.200362 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c57407-8fa7-4018-8c56-90275cbdcae6 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:27.213736 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:27.238651 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cafef3-3afc-48c6-b68e-8fb1e75216a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:27.239264 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cafef3-3afc-48c6-b68e-8fb1e75216a9 None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:27.239598 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cafef3-3afc-48c6-b68e-8fb1e75216a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:27.239808 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cafef3-3afc-48c6-b68e-8fb1e75216a9 None admin] PATH_INFO: `/v3/groups` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:27.240492 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cafef3-3afc-48c6-b68e-8fb1e75216a9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:27.241910 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cafef3-3afc-48c6-b68e-8fb1e75216a9 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:27.261298 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 168/323] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:14:27.261508 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 156/324] 10.222.0.26 () {60 vars in 1134 bytes} [Fri Sep 9 20:14:27 2022] POST /identity/v3/groups => generated 226 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:14:27.411554 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae3a92a-eb73-4196-b62c-fb2bf8b20caa None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:27.411879 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae3a92a-eb73-4196-b62c-fb2bf8b20caa None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:27.412108 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae3a92a-eb73-4196-b62c-fb2bf8b20caa None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:27.413649 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 169/325] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:27.419776 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2038f204-cbb6-459b-8ff7-581346b72db9 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:27.420054 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2038f204-cbb6-459b-8ff7-581346b72db9 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:27.420325 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2038f204-cbb6-459b-8ff7-581346b72db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:27.440208 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-2038f204-cbb6-459b-8ff7-581346b72db9 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:27.441469 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-2038f204-cbb6-459b-8ff7-581346b72db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:27.513894 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 157/326] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:27.534016 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae3a92a-eb73-4196-b62c-fb2bf8b20caa None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:27.650689 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae3a92a-eb73-4196-b62c-fb2bf8b20caa None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:27.695157 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5750108-4c2d-4b77-a7a8-8776188a82d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:27.695758 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5750108-4c2d-4b77-a7a8-8776188a82d0 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:27.696051 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5750108-4c2d-4b77-a7a8-8776188a82d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:27.696323 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5750108-4c2d-4b77-a7a8-8776188a82d0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:27.707194 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5750108-4c2d-4b77-a7a8-8776188a82d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:27.712274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-b5750108-4c2d-4b77-a7a8-8776188a82d0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:14:27.712274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:27.712274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:27.712274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:27.712274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:27.712274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:27.712274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:27.712274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:27.712274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:27.712274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:27.712274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:27.712274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:27.712274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:27.712274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:27.712274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:27.712274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:27.713160 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:27.713160 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:27.713160 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:27.713160 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:27.713160 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:27.713160 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:27.713160 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:27.713160 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:27.713160 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:27.713160 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:27.713160 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:27.713160 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:27.713160 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:27.713160 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:27.713160 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:27.713160 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:27.714004 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:27.714004 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:27.714004 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:27.714004 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:27.714004 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:27.714004 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:27.714004 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:27.714004 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:27.714004 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:27.714004 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:27.714004 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:14:27.714004 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:27.714004 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 170/327] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:14:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:27.719422 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2038f204-cbb6-459b-8ff7-581346b72db9 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:27.743269 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:27.795949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5750108-4c2d-4b77-a7a8-8776188a82d0 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:27.826218 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:27.840642 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2038f204-cbb6-459b-8ff7-581346b72db9 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:27.878796 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9ee8a-a5be-4727-9b2b-60c08260d377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:27.879668 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9ee8a-a5be-4727-9b2b-60c08260d377 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:27.879944 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9ee8a-a5be-4727-9b2b-60c08260d377 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:27.880213 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9ee8a-a5be-4727-9b2b-60c08260d377 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:27.880875 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9ee8a-a5be-4727-9b2b-60c08260d377 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:27.881172 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9ee8a-a5be-4727-9b2b-60c08260d377 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:27.882562 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9ee8a-a5be-4727-9b2b-60c08260d377 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:27.894470 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 158/328] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Sep 9 20:14:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:27.902321 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9ee8a-a5be-4727-9b2b-60c08260d377 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:27.945000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5750108-4c2d-4b77-a7a8-8776188a82d0 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:27.988182 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f1d95-8192-40ef-9720-b0a42e3c18eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:27.989130 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f1d95-8192-40ef-9720-b0a42e3c18eb None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:27.989576 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f1d95-8192-40ef-9720-b0a42e3c18eb None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:27.989992 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f1d95-8192-40ef-9720-b0a42e3c18eb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:28.004526 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:28.006696 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846f1d95-8192-40ef-9720-b0a42e3c18eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:28.011951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-846f1d95-8192-40ef-9720-b0a42e3c18eb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:14:28.011951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:28.011951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:28.011951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:28.011951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:28.011951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:28.011951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:28.011951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:28.011951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:28.011951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:28.011951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:28.011951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:28.011951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:28.011951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:28.011951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:28.011951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:28.012873 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:28.012873 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:28.012873 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:28.012873 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:28.012873 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:28.012873 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:28.012873 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:28.012873 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:28.012873 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:28.012873 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:28.012873 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:28.012873 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:28.012873 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:28.012873 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:28.012873 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:28.012873 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:28.013741 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:28.013741 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:28.013741 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:28.013741 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:28.013741 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:28.013741 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:28.013741 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:28.013741 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:28.013741 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:28.013741 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:28.013741 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:14:28.013741 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:28.013741 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 171/329] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:14:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:14:28.019709 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f1d95-8192-40ef-9720-b0a42e3c18eb None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:28.044198 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9ee8a-a5be-4727-9b2b-60c08260d377 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:28.087965 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a27a3da-f47e-474a-886f-2cc89dfff3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:28.088600 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27a3da-f47e-474a-886f-2cc89dfff3d5 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:28.088875 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27a3da-f47e-474a-886f-2cc89dfff3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:28.089145 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27a3da-f47e-474a-886f-2cc89dfff3d5 None admin] PATH_INFO: `/v3/users/bc399701980047c39d834b377a0233e4` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:28.093035 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a27a3da-f47e-474a-886f-2cc89dfff3d5 None admin] RBAC: Authorizing `identity:get_user(user_id=bc399701980047c39d834b377a0233e4)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:28.094525 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a27a3da-f47e-474a-886f-2cc89dfff3d5 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:28.096480 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 159/330] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:14:27 2022] GET /identity/v3/users/bc399701980047c39d834b377a0233e4 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:28.101660 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a27a3da-f47e-474a-886f-2cc89dfff3d5 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:28.128232 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:28.153570 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f1d95-8192-40ef-9720-b0a42e3c18eb None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:28.195006 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a4037-c166-4280-94f6-5340b03bbb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:28.195651 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a4037-c166-4280-94f6-5340b03bbb6f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:28.195924 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a4037-c166-4280-94f6-5340b03bbb6f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:28.196189 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a4037-c166-4280-94f6-5340b03bbb6f None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:28.196906 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a4037-c166-4280-94f6-5340b03bbb6f None admin] RBAC: Adding query filter params (name=member) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:28.197237 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a4037-c166-4280-94f6-5340b03bbb6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:28.199464 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a4037-c166-4280-94f6-5340b03bbb6f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:28.209880 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 172/331] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Sep 9 20:14:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:28.216856 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a4037-c166-4280-94f6-5340b03bbb6f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:28.225400 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a27a3da-f47e-474a-886f-2cc89dfff3d5 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:28.267737 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687460b1-2fc4-4ebb-bc8d-57162334dda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:28.268913 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687460b1-2fc4-4ebb-bc8d-57162334dda7 None admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:28.269779 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687460b1-2fc4-4ebb-bc8d-57162334dda7 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:28.270262 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687460b1-2fc4-4ebb-bc8d-57162334dda7 None admin] PATH_INFO: `/v3/system/users/bc399701980047c39d834b377a0233e4/roles/3d90aea6ba6344ab9e478143d9be44fd` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:28.272278 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687460b1-2fc4-4ebb-bc8d-57162334dda7 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=bc399701980047c39d834b377a0233e4, role_id=3d90aea6ba6344ab9e478143d9be44fd)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:28.274220 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687460b1-2fc4-4ebb-bc8d-57162334dda7 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:28.283921 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 160/332] 10.222.0.26 () {58 vars in 1334 bytes} [Fri Sep 9 20:14:28 2022] PUT /identity/v3/system/users/bc399701980047c39d834b377a0233e4/roles/3d90aea6ba6344ab9e478143d9be44fd => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:14:28.340825 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a4037-c166-4280-94f6-5340b03bbb6f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:28.383832 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d44b49-b9e1-41e9-9d16-e5abdb48f7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:28.384494 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d44b49-b9e1-41e9-9d16-e5abdb48f7ae None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:28.384772 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d44b49-b9e1-41e9-9d16-e5abdb48f7ae None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:28.385046 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d44b49-b9e1-41e9-9d16-e5abdb48f7ae None admin] PATH_INFO: `/v3/users/2882841c27404ce9a9fdbd1b5f010980` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:28.386993 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d44b49-b9e1-41e9-9d16-e5abdb48f7ae None admin] RBAC: Authorizing `identity:get_user(user_id=2882841c27404ce9a9fdbd1b5f010980)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:28.388448 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d44b49-b9e1-41e9-9d16-e5abdb48f7ae None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:28.390226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 173/333] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:14:28 2022] GET /identity/v3/users/2882841c27404ce9a9fdbd1b5f010980 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:28.396483 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687460b1-2fc4-4ebb-bc8d-57162334dda7 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:28.419548 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:28.544919 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687460b1-2fc4-4ebb-bc8d-57162334dda7 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:28.582499 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7285709-d17d-4fea-a460-4028aeabfafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:28.583117 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7285709-d17d-4fea-a460-4028aeabfafb None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:28.583398 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7285709-d17d-4fea-a460-4028aeabfafb None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:28.583673 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7285709-d17d-4fea-a460-4028aeabfafb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:28.584409 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7285709-d17d-4fea-a460-4028aeabfafb None admin] RBAC: Adding query filter params (role.id=63fc659c11144d369a3f3fa684e056ae, user.id=2882841c27404ce9a9fdbd1b5f010980, scope.system=all) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:28.584716 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7285709-d17d-4fea-a460-4028aeabfafb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:28.586226 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7285709-d17d-4fea-a460-4028aeabfafb None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:28.600391 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 161/334] 10.222.0.26 () {56 vars in 1308 bytes} [Fri Sep 9 20:14:28 2022] GET /identity/v3/role_assignments?user.id=2882841c27404ce9a9fdbd1b5f010980&scope.system=all&role.id=63fc659c11144d369a3f3fa684e056ae => generated 510 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:29.767938 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03318a-f526-477b-bd7f-470090539068 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:29.768436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03318a-f526-477b-bd7f-470090539068 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:29.768536 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03318a-f526-477b-bd7f-470090539068 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:29.770081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 174/335] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:29.784625 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87219a24-aa23-4045-a48f-0e1f5a250b7c None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:29.785617 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87219a24-aa23-4045-a48f-0e1f5a250b7c None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:29.786101 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87219a24-aa23-4045-a48f-0e1f5a250b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:29.812626 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-87219a24-aa23-4045-a48f-0e1f5a250b7c None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:29.814207 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-87219a24-aa23-4045-a48f-0e1f5a250b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:29.891658 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:29.899381 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 162/336] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:30.146280 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191b8c1-4c27-44b1-9602-a51e0777b443 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:30.146963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191b8c1-4c27-44b1-9602-a51e0777b443 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:30.147469 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191b8c1-4c27-44b1-9602-a51e0777b443 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:30.149597 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 175/337] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:30.155097 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa56d2-e713-490f-a0f7-ec2d780fd2b0 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:30.155385 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa56d2-e713-490f-a0f7-ec2d780fd2b0 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:30.155657 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa56d2-e713-490f-a0f7-ec2d780fd2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:30.182245 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-e1aa56d2-e713-490f-a0f7-ec2d780fd2b0 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:30.183427 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-e1aa56d2-e713-490f-a0f7-ec2d780fd2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:30.269147 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 163/338] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:30.400497 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191b8c1-4c27-44b1-9602-a51e0777b443 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:30.418132 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:30.532880 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191b8c1-4c27-44b1-9602-a51e0777b443 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:30.576627 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa7359-b422-4771-8037-c4b84359bbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:30.577284 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa7359-b422-4771-8037-c4b84359bbd2 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:30.577562 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa7359-b422-4771-8037-c4b84359bbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:30.577830 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa7359-b422-4771-8037-c4b84359bbd2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:30.589560 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fa7359-b422-4771-8037-c4b84359bbd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:30.594493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-67fa7359-b422-4771-8037-c4b84359bbd2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:14:30.594493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:30.594493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:30.594493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:30.594493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:30.594493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:30.594493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:30.594493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:30.594493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:30.594493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:30.594493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:30.594493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:30.594493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:30.594493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:30.594493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:30.594493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:30.595361 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:30.595361 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:30.595361 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:30.595361 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:30.595361 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:30.595361 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:30.595361 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:30.595361 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:30.595361 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:30.595361 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:30.595361 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:30.595361 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:30.595361 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:30.595361 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:30.595361 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:30.595361 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:30.596163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:30.596163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:30.596163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:30.596163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:30.596163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:30.596163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:30.596163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:30.596163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:30.596163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:30.596163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:30.596163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:14:30.596163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:30.596163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 176/339] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:14:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:30.604122 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa56d2-e713-490f-a0f7-ec2d780fd2b0 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:30.753058 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa7359-b422-4771-8037-c4b84359bbd2 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:30.755031 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa56d2-e713-490f-a0f7-ec2d780fd2b0 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:30.812141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:30.827897 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20056e88-0b18-4c51-bf5b-78b32a77e99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:30.828526 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20056e88-0b18-4c51-bf5b-78b32a77e99f None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:30.828804 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20056e88-0b18-4c51-bf5b-78b32a77e99f None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:30.829079 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20056e88-0b18-4c51-bf5b-78b32a77e99f None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:30.829796 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20056e88-0b18-4c51-bf5b-78b32a77e99f None admin] RBAC: Adding query filter params (name=member) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:30.830146 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20056e88-0b18-4c51-bf5b-78b32a77e99f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:30.831506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20056e88-0b18-4c51-bf5b-78b32a77e99f None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:30.861657 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 164/340] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Sep 9 20:14:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:30.869172 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20056e88-0b18-4c51-bf5b-78b32a77e99f None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:30.944031 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:31.034811 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa7359-b422-4771-8037-c4b84359bbd2 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:31.036802 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20056e88-0b18-4c51-bf5b-78b32a77e99f None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:31.094057 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995cbb9-002f-4377-8e30-8a11650f80a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:31.094671 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995cbb9-002f-4377-8e30-8a11650f80a4 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:31.094945 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995cbb9-002f-4377-8e30-8a11650f80a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:31.095213 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995cbb9-002f-4377-8e30-8a11650f80a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:31.129913 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e748b-a70d-4c56-a7dd-b9a0b9c63c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:31.130548 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e748b-a70d-4c56-a7dd-b9a0b9c63c99 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:31.130824 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e748b-a70d-4c56-a7dd-b9a0b9c63c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:31.131096 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e748b-a70d-4c56-a7dd-b9a0b9c63c99 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:31.154116 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e748b-a70d-4c56-a7dd-b9a0b9c63c99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:31.163735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-a81e748b-a70d-4c56-a7dd-b9a0b9c63c99 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:14:31.163735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:31.163735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:31.163735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:31.163735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:31.163735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:31.163735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:31.163735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:31.163735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:31.163735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:31.163735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:31.163735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:31.163735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:31.163735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:31.163735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:31.163735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:31.165188 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:31.165188 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:31.165188 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:31.165188 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:31.165188 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:31.165188 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:31.165188 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:31.165188 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:31.165188 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:31.165188 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:31.165188 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:31.165188 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:31.165188 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:31.165188 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:31.165188 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:31.165188 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:31.166018 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:31.166018 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:31.166018 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:31.166018 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:31.166018 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:31.166018 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:31.166018 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:31.166018 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:31.166018 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:31.166018 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:31.166018 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:14:31.166018 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:31.166018 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 177/341] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Sep 9 20:14:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:14:31.167681 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8373e-2306-4b6a-9f4a-94efc8326ab1 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:31.168077 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8373e-2306-4b6a-9f4a-94efc8326ab1 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:31.168363 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8373e-2306-4b6a-9f4a-94efc8326ab1 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:31.174039 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 178/342] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:31.175546 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b8373e-2306-4b6a-9f4a-94efc8326ab1 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:31.196703 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:31.223186 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0995cbb9-002f-4377-8e30-8a11650f80a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:31.225183 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0995cbb9-002f-4377-8e30-8a11650f80a4 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:31.366740 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b8373e-2306-4b6a-9f4a-94efc8326ab1 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:31.413174 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 165/343] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:14:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 20:14:31.416246 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586c7ac-6ebf-4bad-bde0-64c07889a56b None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:31.417399 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586c7ac-6ebf-4bad-bde0-64c07889a56b None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:31.418021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586c7ac-6ebf-4bad-bde0-64c07889a56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:31.422923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:31.433523 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a79a0d-aa79-42b7-b173-8e20c566f33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:31.434461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a79a0d-aa79-42b7-b173-8e20c566f33b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:31.434935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a79a0d-aa79-42b7-b173-8e20c566f33b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:31.436116 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a79a0d-aa79-42b7-b173-8e20c566f33b None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:31.436980 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a79a0d-aa79-42b7-b173-8e20c566f33b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:31.437714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a79a0d-aa79-42b7-b173-8e20c566f33b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:31.439281 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a79a0d-aa79-42b7-b173-8e20c566f33b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:31.454759 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-7586c7ac-6ebf-4bad-bde0-64c07889a56b None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:31.457261 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7586c7ac-6ebf-4bad-bde0-64c07889a56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:31.459554 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 179/344] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:14:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:31.460568 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a79a0d-aa79-42b7-b173-8e20c566f33b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:31.590462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 166/345] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:31.592628 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586c7ac-6ebf-4bad-bde0-64c07889a56b None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:31.638440 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:31.675937 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a79a0d-aa79-42b7-b173-8e20c566f33b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:31.746529 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54e4ec-0974-4bd9-b323-d16fd153ce76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:31.747756 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54e4ec-0974-4bd9-b323-d16fd153ce76 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:31.749331 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54e4ec-0974-4bd9-b323-d16fd153ce76 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:31.751201 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54e4ec-0974-4bd9-b323-d16fd153ce76 None admin] PATH_INFO: `/v3/projects/462d8ea458304464846aba18be1b8665` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:31.752596 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c54e4ec-0974-4bd9-b323-d16fd153ce76 None admin] RBAC: Authorizing `identity:get_project(project_id=462d8ea458304464846aba18be1b8665)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:31.754668 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c54e4ec-0974-4bd9-b323-d16fd153ce76 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:31.757432 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 180/346] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:14:31 2022] GET /identity/v3/projects/462d8ea458304464846aba18be1b8665 => generated 298 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:31.762801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54e4ec-0974-4bd9-b323-d16fd153ce76 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:31.800535 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586c7ac-6ebf-4bad-bde0-64c07889a56b None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:31.854010 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:31.866627 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0d9d0-c587-4535-aa89-f6652043fe80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:31.867323 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0d9d0-c587-4535-aa89-f6652043fe80 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:31.867531 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0d9d0-c587-4535-aa89-f6652043fe80 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:31.867802 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0d9d0-c587-4535-aa89-f6652043fe80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:31.925840 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54e4ec-0974-4bd9-b323-d16fd153ce76 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:31.971084 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb0d9d0-c587-4535-aa89-f6652043fe80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:31.972588 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb0d9d0-c587-4535-aa89-f6652043fe80 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:31.995149 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef345c2f-26d9-4bcf-9719-6e78032f9c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:31.995827 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef345c2f-26d9-4bcf-9719-6e78032f9c76 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:31.996069 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef345c2f-26d9-4bcf-9719-6e78032f9c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:31.996337 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef345c2f-26d9-4bcf-9719-6e78032f9c76 None admin] PATH_INFO: `/v3/groups/685ef4c7d36e45e5961212c7d6e6c614` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:32.001388 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:32.002665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef345c2f-26d9-4bcf-9719-6e78032f9c76 None admin] RBAC: Authorizing `identity:get_group(group_id=685ef4c7d36e45e5961212c7d6e6c614)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:32.004469 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef345c2f-26d9-4bcf-9719-6e78032f9c76 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:32.006175 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 181/347] 10.222.0.26 () {56 vars in 1180 bytes} [Fri Sep 9 20:14:31 2022] GET /identity/v3/groups/685ef4c7d36e45e5961212c7d6e6c614 => generated 226 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:32.007327 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842bade-d650-4712-89ef-98edf7aac547 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:32.008129 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842bade-d650-4712-89ef-98edf7aac547 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:32.008406 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842bade-d650-4712-89ef-98edf7aac547 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:32.015200 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 182/348] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:32.015997 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7842bade-d650-4712-89ef-98edf7aac547 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:32.102806 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 167/349] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:14:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:14:32.104115 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0d9d0-c587-4535-aa89-f6652043fe80 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:32.194928 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7842bade-d650-4712-89ef-98edf7aac547 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:32.232452 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:32.238228 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0d9d0-c587-4535-aa89-f6652043fe80 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:32.245267 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f8e8b-3e67-412b-ad9f-2b323db9744e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:32.245904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f8e8b-3e67-412b-ad9f-2b323db9744e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:32.248132 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f8e8b-3e67-412b-ad9f-2b323db9744e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:32.248132 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f8e8b-3e67-412b-ad9f-2b323db9744e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:32.260909 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35f8e8b-3e67-412b-ad9f-2b323db9744e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:32.266298 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-e35f8e8b-3e67-412b-ad9f-2b323db9744e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:14:32.266298 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:32.266298 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:32.266298 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:32.266298 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:32.266298 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:32.266298 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:32.266298 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:32.266298 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:32.266298 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:32.266298 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:32.266298 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:32.266298 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:32.266298 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:32.266298 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:32.266298 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:32.267256 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:32.267256 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:32.267256 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:32.267256 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:32.267256 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:32.267256 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:32.267256 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:32.267256 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:32.267256 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:32.267256 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:32.267256 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:32.267256 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:32.267256 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:32.267256 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:32.267256 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:32.267256 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:32.268483 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:32.268483 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:32.268483 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:32.268483 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:32.268483 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:32.268483 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:32.268483 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:32.268483 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:32.268483 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:32.268483 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:32.268483 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:14:32.268483 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:32.268483 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 183/350] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:14:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:32.269203 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd11cf8-cc25-4a11-9713-f0c22820f0b5 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:32.271052 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd11cf8-cc25-4a11-9713-f0c22820f0b5 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:32.271052 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd11cf8-cc25-4a11-9713-f0c22820f0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:32.274635 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:32.290810 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8b9c8-f6e8-4e42-8914-e0cc0e53213d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:32.291430 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8b9c8-f6e8-4e42-8914-e0cc0e53213d None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:32.291711 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8b9c8-f6e8-4e42-8914-e0cc0e53213d None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:32.291982 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8b9c8-f6e8-4e42-8914-e0cc0e53213d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:32.292713 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8b9c8-f6e8-4e42-8914-e0cc0e53213d None admin] RBAC: Adding query filter params (group.id=685ef4c7d36e45e5961212c7d6e6c614, role.id=63fc659c11144d369a3f3fa684e056ae, scope.project.id=462d8ea458304464846aba18be1b8665) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:32.293018 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8b9c8-f6e8-4e42-8914-e0cc0e53213d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:32.294513 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8b9c8-f6e8-4e42-8914-e0cc0e53213d None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:32.294783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-cfd11cf8-cc25-4a11-9713-f0c22820f0b5 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:32.295987 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-cfd11cf8-cc25-4a11-9713-f0c22820f0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:32.301532 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 168/351] 10.222.0.26 () {56 vars in 1376 bytes} [Fri Sep 9 20:14:32 2022] GET /identity/v3/role_assignments?group.id=685ef4c7d36e45e5961212c7d6e6c614&scope.project.id=462d8ea458304464846aba18be1b8665&role.id=63fc659c11144d369a3f3fa684e056ae => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:32.302149 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8b9c8-f6e8-4e42-8914-e0cc0e53213d None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:32.397932 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 184/352] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:32.401470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd11cf8-cc25-4a11-9713-f0c22820f0b5 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:32.438758 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8b9c8-f6e8-4e42-8914-e0cc0e53213d None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:32.512870 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c3b26-afee-4b56-b035-d685d66d4259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:32.513468 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c3b26-afee-4b56-b035-d685d66d4259 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:32.513752 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c3b26-afee-4b56-b035-d685d66d4259 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:32.514027 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c3b26-afee-4b56-b035-d685d66d4259 None admin] PATH_INFO: `/v3/projects/462d8ea458304464846aba18be1b8665` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:32.515277 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c3b26-afee-4b56-b035-d685d66d4259 None admin] RBAC: Authorizing `identity:get_project(project_id=462d8ea458304464846aba18be1b8665)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:32.516718 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c3b26-afee-4b56-b035-d685d66d4259 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:32.518383 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 169/353] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:14:32 2022] GET /identity/v3/projects/462d8ea458304464846aba18be1b8665 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:32.519516 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f733b2-8002-4cf7-831d-44627b4446a4 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:32.519802 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f733b2-8002-4cf7-831d-44627b4446a4 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:32.520066 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f733b2-8002-4cf7-831d-44627b4446a4 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:32.521283 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 170/354] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:32.526548 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f733b2-8002-4cf7-831d-44627b4446a4 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:32.589154 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd11cf8-cc25-4a11-9713-f0c22820f0b5 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:32.603644 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:32.611301 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:32.636920 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfc4bb-111a-412a-817e-0218eed2e1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:32.637713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfc4bb-111a-412a-817e-0218eed2e1cd None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:32.637989 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfc4bb-111a-412a-817e-0218eed2e1cd None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:32.638258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfc4bb-111a-412a-817e-0218eed2e1cd None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:32.638924 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dfc4bb-111a-412a-817e-0218eed2e1cd None admin] RBAC: Adding query filter params (name=member) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:32.639222 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dfc4bb-111a-412a-817e-0218eed2e1cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:32.640805 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dfc4bb-111a-412a-817e-0218eed2e1cd None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:32.650197 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 185/355] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Sep 9 20:14:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:32.651575 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930cdb9-f663-44e3-9f78-55c2e42f5d37 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:32.651859 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930cdb9-f663-44e3-9f78-55c2e42f5d37 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:32.652130 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930cdb9-f663-44e3-9f78-55c2e42f5d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:32.676038 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f733b2-8002-4cf7-831d-44627b4446a4 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:32.678034 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f930cdb9-f663-44e3-9f78-55c2e42f5d37 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:32.679338 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f930cdb9-f663-44e3-9f78-55c2e42f5d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:32.725316 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c14f1-e12f-448f-acf3-a1474b2cdecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:32.727127 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c14f1-e12f-448f-acf3-a1474b2cdecb None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:32.727623 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c14f1-e12f-448f-acf3-a1474b2cdecb None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:32.729020 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c14f1-e12f-448f-acf3-a1474b2cdecb None admin] PATH_INFO: `/v3/groups/685ef4c7d36e45e5961212c7d6e6c614` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:32.730403 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c14f1-e12f-448f-acf3-a1474b2cdecb None admin] RBAC: Authorizing `identity:get_group(group_id=685ef4c7d36e45e5961212c7d6e6c614)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:32.732112 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c14f1-e12f-448f-acf3-a1474b2cdecb None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:32.734853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 171/356] 10.222.0.26 () {56 vars in 1180 bytes} [Fri Sep 9 20:14:32 2022] GET /identity/v3/groups/685ef4c7d36e45e5961212c7d6e6c614 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:32.736718 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391d5f2-d3be-48ae-b491-b6302d13837d None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:32.737644 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391d5f2-d3be-48ae-b491-b6302d13837d None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:32.738142 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391d5f2-d3be-48ae-b491-b6302d13837d None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:32.740387 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 172/357] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:32.741852 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391d5f2-d3be-48ae-b491-b6302d13837d None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:32.770588 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 186/358] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:32.771205 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f930cdb9-f663-44e3-9f78-55c2e42f5d37 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:32.880712 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391d5f2-d3be-48ae-b491-b6302d13837d None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:32.905676 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:32.923887 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f930cdb9-f663-44e3-9f78-55c2e42f5d37 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:32.924970 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a13fea-b7ad-4c89-bc08-adff37ac08ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:32.925609 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a13fea-b7ad-4c89-bc08-adff37ac08ea None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:32.925886 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a13fea-b7ad-4c89-bc08-adff37ac08ea None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:32.926154 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a13fea-b7ad-4c89-bc08-adff37ac08ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:32.936917 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:32.979805 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60ea9f-cd73-4653-9ff4-856e2e9a82a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:32.979805 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60ea9f-cd73-4653-9ff4-856e2e9a82a4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:32.979805 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60ea9f-cd73-4653-9ff4-856e2e9a82a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:32.979805 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60ea9f-cd73-4653-9ff4-856e2e9a82a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:32.980225 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d60ea9f-cd73-4653-9ff4-856e2e9a82a4 None admin] RBAC: Adding query filter params (group.id=685ef4c7d36e45e5961212c7d6e6c614, role.id=d5528405b19b48a6bc4b7ff451332d8a, scope.project.id=462d8ea458304464846aba18be1b8665) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:32.980225 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d60ea9f-cd73-4653-9ff4-856e2e9a82a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:32.980802 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d60ea9f-cd73-4653-9ff4-856e2e9a82a4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:32.988856 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 187/359] 10.222.0.26 () {56 vars in 1376 bytes} [Fri Sep 9 20:14:32 2022] GET /identity/v3/role_assignments?group.id=685ef4c7d36e45e5961212c7d6e6c614&scope.project.id=462d8ea458304464846aba18be1b8665&role.id=d5528405b19b48a6bc4b7ff451332d8a => generated 260 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:32.990405 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8090ab8-7901-4a17-b405-f4616200e50b None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:32.990749 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8090ab8-7901-4a17-b405-f4616200e50b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:32.991026 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8090ab8-7901-4a17-b405-f4616200e50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:33.020050 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f8090ab8-7901-4a17-b405-f4616200e50b None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:33.021536 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f8090ab8-7901-4a17-b405-f4616200e50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:33.022024 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a13fea-b7ad-4c89-bc08-adff37ac08ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:33.024434 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a13fea-b7ad-4c89-bc08-adff37ac08ea None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:33.121971 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 188/360] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:33.122847 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8090ab8-7901-4a17-b405-f4616200e50b None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:33.152991 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 173/361] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:14:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:14:33.154323 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a13fea-b7ad-4c89-bc08-adff37ac08ea None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:33.179639 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:33.273632 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8090ab8-7901-4a17-b405-f4616200e50b None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:33.277575 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:33.324119 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a999d-c8e7-45bc-a3e5-8a742ca02782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:33.324757 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a999d-c8e7-45bc-a3e5-8a742ca02782 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:33.325033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a999d-c8e7-45bc-a3e5-8a742ca02782 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:33.325326 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a999d-c8e7-45bc-a3e5-8a742ca02782 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:33.332039 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a13fea-b7ad-4c89-bc08-adff37ac08ea None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:33.347673 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4a999d-c8e7-45bc-a3e5-8a742ca02782 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:33.352713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-fc4a999d-c8e7-45bc-a3e5-8a742ca02782 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:14:33.352713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:33.352713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:33.352713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:33.352713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:33.352713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:33.352713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:33.352713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:33.352713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:33.352713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:33.352713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:33.352713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:33.352713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:33.352713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:33.352713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:33.352713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:33.353714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:33.353714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:33.353714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:33.353714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:33.353714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:33.353714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:33.353714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:33.353714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:33.353714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:33.353714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:33.353714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:33.353714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:33.353714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:33.353714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:33.353714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:33.353714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:33.354528 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:33.354528 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:33.354528 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:33.354528 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:33.354528 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:33.354528 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:33.354528 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:33.354528 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:33.354528 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:33.354528 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:33.354528 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:14:33.354528 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:33.354528 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 189/362] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:14:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:33.355218 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a999d-c8e7-45bc-a3e5-8a742ca02782 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:33.386564 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd63180-d2cb-4de2-bc34-6601e1f2467c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:33.387201 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd63180-d2cb-4de2-bc34-6601e1f2467c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:33.387478 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd63180-d2cb-4de2-bc34-6601e1f2467c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:33.390472 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd63180-d2cb-4de2-bc34-6601e1f2467c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:33.410848 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd63180-d2cb-4de2-bc34-6601e1f2467c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:33.416557 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-1dd63180-d2cb-4de2-bc34-6601e1f2467c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:14:33.416557 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:33.416557 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:33.416557 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:33.416557 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:33.416557 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:33.416557 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:33.416557 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:33.416557 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:33.416557 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:33.416557 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:33.416557 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:33.416557 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:33.416557 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:33.416557 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:33.416557 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:33.418709 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:33.418709 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:33.418709 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:33.418709 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:33.418709 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:33.418709 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:33.418709 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:33.418709 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:33.418709 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:33.418709 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:33.418709 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:33.418709 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:33.418709 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:33.418709 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:33.418709 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:33.418709 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:33.420732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:33.420732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:33.420732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:33.420732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:33.420732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:33.420732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:33.420732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:33.420732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:33.420732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:33.420732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:33.420732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:14:33.420732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:14:33.420732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 174/363] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Sep 9 20:14:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:33.420732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd63180-d2cb-4de2-bc34-6601e1f2467c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:33.498115 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a999d-c8e7-45bc-a3e5-8a742ca02782 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:33.536464 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38414529-483c-48ab-98e4-62a8c645bb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:33.537028 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38414529-483c-48ab-98e4-62a8c645bb2c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:33.537326 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38414529-483c-48ab-98e4-62a8c645bb2c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:33.537598 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38414529-483c-48ab-98e4-62a8c645bb2c None admin] PATH_INFO: `/v3/projects/445158de5da046e19c309424d250c953` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:33.540275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38414529-483c-48ab-98e4-62a8c645bb2c None admin] RBAC: Authorizing `identity:get_project(project_id=445158de5da046e19c309424d250c953)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:33.541732 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38414529-483c-48ab-98e4-62a8c645bb2c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:33.543525 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 190/364] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:14:33 2022] GET /identity/v3/projects/445158de5da046e19c309424d250c953 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:33.544122 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38414529-483c-48ab-98e4-62a8c645bb2c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:33.575202 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:33.578884 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:33.597489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd63180-d2cb-4de2-bc34-6601e1f2467c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:33.635739 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93fd18-ef5d-4184-ac29-6f3f8beb0c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:33.636345 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93fd18-ef5d-4184-ac29-6f3f8beb0c8c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:33.636620 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93fd18-ef5d-4184-ac29-6f3f8beb0c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:33.636892 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93fd18-ef5d-4184-ac29-6f3f8beb0c8c None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:33.637600 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93fd18-ef5d-4184-ac29-6f3f8beb0c8c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:33.637896 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93fd18-ef5d-4184-ac29-6f3f8beb0c8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:33.639278 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93fd18-ef5d-4184-ac29-6f3f8beb0c8c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:33.647939 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 175/365] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Sep 9 20:14:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:33.648774 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93fd18-ef5d-4184-ac29-6f3f8beb0c8c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:33.652588 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38414529-483c-48ab-98e4-62a8c645bb2c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:33.707908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9379d-6edf-4b4f-bc4a-c969c5347eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:33.708482 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9379d-6edf-4b4f-bc4a-c969c5347eb5 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:33.708742 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9379d-6edf-4b4f-bc4a-c969c5347eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:33.709009 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9379d-6edf-4b4f-bc4a-c969c5347eb5 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:33.709723 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9379d-6edf-4b4f-bc4a-c969c5347eb5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:33.710025 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9379d-6edf-4b4f-bc4a-c969c5347eb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:33.711469 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9379d-6edf-4b4f-bc4a-c969c5347eb5 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:33.722316 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 191/366] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:14:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:33.722785 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9379d-6edf-4b4f-bc4a-c969c5347eb5 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:33.784577 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93fd18-ef5d-4184-ac29-6f3f8beb0c8c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:33.831136 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5bed25-568d-4e74-8a31-8a4483f764d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:33.831136 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5bed25-568d-4e74-8a31-8a4483f764d5 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:33.831374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5bed25-568d-4e74-8a31-8a4483f764d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:33.831445 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5bed25-568d-4e74-8a31-8a4483f764d5 None admin] PATH_INFO: `/v3/groups/685ef4c7d36e45e5961212c7d6e6c614` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:33.832628 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5bed25-568d-4e74-8a31-8a4483f764d5 None admin] RBAC: Authorizing `identity:get_group(group_id=685ef4c7d36e45e5961212c7d6e6c614)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:33.834651 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5bed25-568d-4e74-8a31-8a4483f764d5 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:33.837576 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 176/367] 10.222.0.26 () {56 vars in 1180 bytes} [Fri Sep 9 20:14:33 2022] GET /identity/v3/groups/685ef4c7d36e45e5961212c7d6e6c614 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:33.838398 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5bed25-568d-4e74-8a31-8a4483f764d5 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:33.854017 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:33.855545 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:33.865400 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9379d-6edf-4b4f-bc4a-c969c5347eb5 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:33.913914 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80709b88-92a6-4d18-95ec-5b2d470e480f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:33.914922 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80709b88-92a6-4d18-95ec-5b2d470e480f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:33.915427 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80709b88-92a6-4d18-95ec-5b2d470e480f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:33.918366 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80709b88-92a6-4d18-95ec-5b2d470e480f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:33.930051 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80709b88-92a6-4d18-95ec-5b2d470e480f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:33.938749 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-80709b88-92a6-4d18-95ec-5b2d470e480f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:14:33.938749 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:33.938749 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:33.938749 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:33.938749 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:33.938749 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:33.938749 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:33.938749 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:33.938749 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:33.938749 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:33.938749 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:33.938749 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:33.938749 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:33.938749 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:33.938749 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:33.938749 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:33.939678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:33.939678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:33.939678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:33.939678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:33.939678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:33.939678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:33.939678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:33.939678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:33.939678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:33.939678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:33.939678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:33.939678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:33.939678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:33.939678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:33.939678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:33.939678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:33.940493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:33.940493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:33.940493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:33.940493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:33.940493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:33.940493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:33.940493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:33.940493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:33.940493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:33.940493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:33.940493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:14:33.940493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:33.943111 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 192/368] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:14:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:33.944094 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80709b88-92a6-4d18-95ec-5b2d470e480f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:33.969800 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5bed25-568d-4e74-8a31-8a4483f764d5 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:34.010043 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ccb80c-67c1-4df2-9dac-4f3d2c372eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:34.010802 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ccb80c-67c1-4df2-9dac-4f3d2c372eee None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:34.011148 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ccb80c-67c1-4df2-9dac-4f3d2c372eee None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:34.011496 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ccb80c-67c1-4df2-9dac-4f3d2c372eee None admin] PATH_INFO: `/v3/users/bc399701980047c39d834b377a0233e4` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:34.012667 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ccb80c-67c1-4df2-9dac-4f3d2c372eee None admin] RBAC: Authorizing `identity:get_user(user_id=bc399701980047c39d834b377a0233e4)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:34.014449 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ccb80c-67c1-4df2-9dac-4f3d2c372eee None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:34.016613 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 177/369] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:14:33 2022] GET /identity/v3/users/bc399701980047c39d834b377a0233e4 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:34.018812 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ccb80c-67c1-4df2-9dac-4f3d2c372eee None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:34.061404 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80709b88-92a6-4d18-95ec-5b2d470e480f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:34.098186 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f0840-5b4f-4400-bb1f-d39050593919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:34.099134 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f0840-5b4f-4400-bb1f-d39050593919 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:34.099641 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f0840-5b4f-4400-bb1f-d39050593919 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:34.100154 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f0840-5b4f-4400-bb1f-d39050593919 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:34.108420 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:34.125804 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:34.136812 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ccb80c-67c1-4df2-9dac-4f3d2c372eee None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:34.174850 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3a059-2960-4127-9488-334bc1ef03fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:34.176084 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3a059-2960-4127-9488-334bc1ef03fe None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:34.176441 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3a059-2960-4127-9488-334bc1ef03fe None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:34.176787 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3a059-2960-4127-9488-334bc1ef03fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:34.177635 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d3a059-2960-4127-9488-334bc1ef03fe None admin] RBAC: Adding query filter params (group.id=685ef4c7d36e45e5961212c7d6e6c614, role.id=63fc659c11144d369a3f3fa684e056ae, scope.project.id=445158de5da046e19c309424d250c953) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:34.178012 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d3a059-2960-4127-9488-334bc1ef03fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:34.179779 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d3a059-2960-4127-9488-334bc1ef03fe None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:34.183647 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f0840-5b4f-4400-bb1f-d39050593919 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:34.185338 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f0840-5b4f-4400-bb1f-d39050593919 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:34.187968 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 178/370] 10.222.0.26 () {56 vars in 1376 bytes} [Fri Sep 9 20:14:34 2022] GET /identity/v3/role_assignments?group.id=685ef4c7d36e45e5961212c7d6e6c614&scope.project.id=445158de5da046e19c309424d250c953&role.id=63fc659c11144d369a3f3fa684e056ae => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:34.193324 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3a059-2960-4127-9488-334bc1ef03fe None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:34.373541 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 193/371] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:14:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:14:34.374501 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f0840-5b4f-4400-bb1f-d39050593919 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:34.447057 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3a059-2960-4127-9488-334bc1ef03fe None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:34.459148 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:34.517546 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ce58a-71ba-4f4c-b723-18434a57e2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:34.518240 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ce58a-71ba-4f4c-b723-18434a57e2c2 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:34.518631 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ce58a-71ba-4f4c-b723-18434a57e2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:34.518966 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ce58a-71ba-4f4c-b723-18434a57e2c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:34.519832 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ce58a-71ba-4f4c-b723-18434a57e2c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:34.520148 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ce58a-71ba-4f4c-b723-18434a57e2c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:34.522318 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ce58a-71ba-4f4c-b723-18434a57e2c2 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:34.540395 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 179/372] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:14:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:34.546485 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ce58a-71ba-4f4c-b723-18434a57e2c2 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:34.556204 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:34.582073 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f0840-5b4f-4400-bb1f-d39050593919 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:34.674098 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e58eddd-9b0b-4f1d-98ed-6cc5e43b7364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:34.674832 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58eddd-9b0b-4f1d-98ed-6cc5e43b7364 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:34.675162 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58eddd-9b0b-4f1d-98ed-6cc5e43b7364 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:34.675505 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58eddd-9b0b-4f1d-98ed-6cc5e43b7364 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:34.676325 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e58eddd-9b0b-4f1d-98ed-6cc5e43b7364 None admin] RBAC: Adding query filter params (role.id=3d90aea6ba6344ab9e478143d9be44fd, user.id=bc399701980047c39d834b377a0233e4, scope.system=all) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:34.676708 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e58eddd-9b0b-4f1d-98ed-6cc5e43b7364 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:34.679614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e58eddd-9b0b-4f1d-98ed-6cc5e43b7364 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:34.702709 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 194/373] 10.222.0.26 () {56 vars in 1308 bytes} [Fri Sep 9 20:14:34 2022] GET /identity/v3/role_assignments?user.id=bc399701980047c39d834b377a0233e4&scope.system=all&role.id=3d90aea6ba6344ab9e478143d9be44fd => generated 510 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:34.705126 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e58eddd-9b0b-4f1d-98ed-6cc5e43b7364 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:34.755269 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ce58a-71ba-4f4c-b723-18434a57e2c2 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:34.866630 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7b8f5-6af0-4c5f-bb23-85f70de21e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:34.867319 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7b8f5-6af0-4c5f-bb23-85f70de21e97 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:34.867602 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7b8f5-6af0-4c5f-bb23-85f70de21e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:34.867878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7b8f5-6af0-4c5f-bb23-85f70de21e97 None admin] PATH_INFO: `/v3/projects/445158de5da046e19c309424d250c953` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:34.869031 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b7b8f5-6af0-4c5f-bb23-85f70de21e97 None admin] RBAC: Authorizing `identity:get_project(project_id=445158de5da046e19c309424d250c953)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:34.870560 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b7b8f5-6af0-4c5f-bb23-85f70de21e97 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:34.872174 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 180/374] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:14:34 2022] GET /identity/v3/projects/445158de5da046e19c309424d250c953 => generated 302 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:34.877817 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7b8f5-6af0-4c5f-bb23-85f70de21e97 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:34.894977 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e58eddd-9b0b-4f1d-98ed-6cc5e43b7364 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:34.910279 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:34.986667 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba6af78-4651-4e03-8004-cdff7e9f508d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:34.987302 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6af78-4651-4e03-8004-cdff7e9f508d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:34.987581 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6af78-4651-4e03-8004-cdff7e9f508d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:34.987849 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6af78-4651-4e03-8004-cdff7e9f508d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:35.017476 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:35.062947 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7b8f5-6af0-4c5f-bb23-85f70de21e97 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:35.108249 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b979b19c-6e2a-4a7b-a0b2-f9c70edd56c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:35.109127 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979b19c-6e2a-4a7b-a0b2-f9c70edd56c0 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:35.109563 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979b19c-6e2a-4a7b-a0b2-f9c70edd56c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:35.109971 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979b19c-6e2a-4a7b-a0b2-f9c70edd56c0 None admin] PATH_INFO: `/v3/groups/685ef4c7d36e45e5961212c7d6e6c614` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:35.111499 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b979b19c-6e2a-4a7b-a0b2-f9c70edd56c0 None admin] RBAC: Authorizing `identity:get_group(group_id=685ef4c7d36e45e5961212c7d6e6c614)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:35.113874 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b979b19c-6e2a-4a7b-a0b2-f9c70edd56c0 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:35.114789 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba6af78-4651-4e03-8004-cdff7e9f508d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:35.116332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba6af78-4651-4e03-8004-cdff7e9f508d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:35.116642 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 181/375] 10.222.0.26 () {56 vars in 1180 bytes} [Fri Sep 9 20:14:34 2022] GET /identity/v3/groups/685ef4c7d36e45e5961212c7d6e6c614 => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:35.118627 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72ad8b-de2a-45ac-8f99-dba32dc5daa8 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:35.119322 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72ad8b-de2a-45ac-8f99-dba32dc5daa8 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:35.119726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72ad8b-de2a-45ac-8f99-dba32dc5daa8 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:35.122150 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 182/376] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:35.125299 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72ad8b-de2a-45ac-8f99-dba32dc5daa8 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:35.262294 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 195/377] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:14:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:14:35.263543 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229b812-27c9-4c14-8105-8d286f08a8a8 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:35.263820 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229b812-27c9-4c14-8105-8d286f08a8a8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:35.264096 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229b812-27c9-4c14-8105-8d286f08a8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:35.266946 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72ad8b-de2a-45ac-8f99-dba32dc5daa8 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:35.290846 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a229b812-27c9-4c14-8105-8d286f08a8a8 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:35.292064 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a229b812-27c9-4c14-8105-8d286f08a8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:35.301725 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:35.334636 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b17e26-52a2-412a-bfcf-b123c2739a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:35.335400 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b17e26-52a2-412a-bfcf-b123c2739a98 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:35.335524 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b17e26-52a2-412a-bfcf-b123c2739a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:35.336353 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b17e26-52a2-412a-bfcf-b123c2739a98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:35.337089 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b17e26-52a2-412a-bfcf-b123c2739a98 None admin] RBAC: Adding query filter params (group.id=685ef4c7d36e45e5961212c7d6e6c614, role.id=d5528405b19b48a6bc4b7ff451332d8a, scope.project.id=445158de5da046e19c309424d250c953) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:35.337426 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b17e26-52a2-412a-bfcf-b123c2739a98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:35.338877 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b17e26-52a2-412a-bfcf-b123c2739a98 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:35.344855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 183/378] 10.222.0.26 () {56 vars in 1376 bytes} [Fri Sep 9 20:14:35 2022] GET /identity/v3/role_assignments?group.id=685ef4c7d36e45e5961212c7d6e6c614&scope.project.id=445158de5da046e19c309424d250c953&role.id=d5528405b19b48a6bc4b7ff451332d8a => generated 260 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:35.345705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b17e26-52a2-412a-bfcf-b123c2739a98 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:35.350870 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:35.397340 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 196/379] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:35.500501 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b17e26-52a2-412a-bfcf-b123c2739a98 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:35.542543 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78342e7-41aa-449d-87d9-62db0e6480f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:35.543114 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78342e7-41aa-449d-87d9-62db0e6480f9 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:35.543391 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78342e7-41aa-449d-87d9-62db0e6480f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:35.543667 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78342e7-41aa-449d-87d9-62db0e6480f9 None admin] PATH_INFO: `/v3/projects/e8a4d19e4885495c84cf29b80391fd52` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:35.546044 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78342e7-41aa-449d-87d9-62db0e6480f9 None admin] RBAC: Authorizing `identity:get_project(project_id=e8a4d19e4885495c84cf29b80391fd52)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:35.547563 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78342e7-41aa-449d-87d9-62db0e6480f9 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:35.549358 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 184/380] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:14:35 2022] GET /identity/v3/projects/e8a4d19e4885495c84cf29b80391fd52 => generated 344 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:35.555613 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a229b812-27c9-4c14-8105-8d286f08a8a8 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:35.688565 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a229b812-27c9-4c14-8105-8d286f08a8a8 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:35.726672 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef0a9d-5f78-46c4-8ac0-017ce96f8653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:35.727452 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef0a9d-5f78-46c4-8ac0-017ce96f8653 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:35.727905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef0a9d-5f78-46c4-8ac0-017ce96f8653 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:35.728273 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef0a9d-5f78-46c4-8ac0-017ce96f8653 None admin] PATH_INFO: `/v3/groups/e4c59e51c2e144c18fc58ca427948854` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:35.735448 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adef0a9d-5f78-46c4-8ac0-017ce96f8653 None admin] RBAC: Authorizing `identity:get_group(group_id=e4c59e51c2e144c18fc58ca427948854)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:35.736990 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adef0a9d-5f78-46c4-8ac0-017ce96f8653 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:35.739087 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 197/381] 10.222.0.26 () {56 vars in 1180 bytes} [Fri Sep 9 20:14:35 2022] GET /identity/v3/groups/e4c59e51c2e144c18fc58ca427948854 => generated 229 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:35.744606 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78342e7-41aa-449d-87d9-62db0e6480f9 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:35.799416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef0a9d-5f78-46c4-8ac0-017ce96f8653 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:35.803334 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:35.846747 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:35.889510 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78342e7-41aa-449d-87d9-62db0e6480f9 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:35.931991 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f00f1c4-669d-478c-b7dc-57cbc302b4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:35.932979 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00f1c4-669d-478c-b7dc-57cbc302b4b8 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:35.934288 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00f1c4-669d-478c-b7dc-57cbc302b4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:35.934288 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00f1c4-669d-478c-b7dc-57cbc302b4b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:35.935081 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f00f1c4-669d-478c-b7dc-57cbc302b4b8 None admin] RBAC: Adding query filter params (group.id=e4c59e51c2e144c18fc58ca427948854, role.id=9121ba25a6d84756990f0c3a19e9fecb, scope.project.id=e8a4d19e4885495c84cf29b80391fd52) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:35.935391 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f00f1c4-669d-478c-b7dc-57cbc302b4b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:35.937015 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f00f1c4-669d-478c-b7dc-57cbc302b4b8 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:35.946321 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef0a9d-5f78-46c4-8ac0-017ce96f8653 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:35.948938 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 185/382] 10.222.0.26 () {56 vars in 1376 bytes} [Fri Sep 9 20:14:35 2022] GET /identity/v3/role_assignments?group.id=e4c59e51c2e144c18fc58ca427948854&scope.project.id=e8a4d19e4885495c84cf29b80391fd52&role.id=9121ba25a6d84756990f0c3a19e9fecb => generated 260 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:35.951017 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11384be0-3ee5-49ba-9c6c-de9af258449a None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:35.951017 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11384be0-3ee5-49ba-9c6c-de9af258449a None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:35.951017 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11384be0-3ee5-49ba-9c6c-de9af258449a None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:35.956250 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 186/383] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:35.959174 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a9660-9d2f-4b70-9c21-4ef13480f59a None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:35.959458 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a9660-9d2f-4b70-9c21-4ef13480f59a None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:35.959694 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a9660-9d2f-4b70-9c21-4ef13480f59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:35.981715 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-d70a9660-9d2f-4b70-9c21-4ef13480f59a None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:35.982939 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-d70a9660-9d2f-4b70-9c21-4ef13480f59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:35.999088 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684518c4-3e8b-4446-850b-9f2f67a82eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:36.000266 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684518c4-3e8b-4446-850b-9f2f67a82eef None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:36.000741 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684518c4-3e8b-4446-850b-9f2f67a82eef None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:36.001391 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684518c4-3e8b-4446-850b-9f2f67a82eef None admin] PATH_INFO: `/v3/roles/member` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:36.016615 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684518c4-3e8b-4446-850b-9f2f67a82eef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:36.022935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-684518c4-3e8b-4446-850b-9f2f67a82eef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:14:36.022935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:36.022935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:36.022935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:36.022935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:36.022935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:36.022935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:36.022935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:36.022935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:36.022935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:36.022935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:36.022935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:36.022935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:36.022935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:36.022935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:36.022935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:36.024500 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:36.024500 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:36.024500 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:36.024500 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:36.024500 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:36.024500 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:36.024500 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:36.024500 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:36.024500 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:36.024500 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:36.024500 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:36.024500 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:36.024500 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:36.024500 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:36.024500 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:36.024500 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:36.025385 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:36.025385 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:36.025385 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:36.025385 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:36.025385 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:36.025385 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:36.025385 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:36.025385 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:36.025385 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:36.025385 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:36.025385 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:14:36.025385 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:36.025385 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 198/384] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:14:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:14:36.032939 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684518c4-3e8b-4446-850b-9f2f67a82eef None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:36.103761 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 187/385] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:36.142423 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:36.198099 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684518c4-3e8b-4446-850b-9f2f67a82eef None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:36.237609 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3706fa-b44c-4503-b839-298b7240b754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:36.238266 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3706fa-b44c-4503-b839-298b7240b754 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:36.238550 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3706fa-b44c-4503-b839-298b7240b754 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:36.238821 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3706fa-b44c-4503-b839-298b7240b754 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:36.239490 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3706fa-b44c-4503-b839-298b7240b754 None admin] RBAC: Adding query filter params (name=member) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:36.239791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3706fa-b44c-4503-b839-298b7240b754 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:36.241157 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3706fa-b44c-4503-b839-298b7240b754 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:36.252518 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 199/386] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Sep 9 20:14:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:36.258327 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a9660-9d2f-4b70-9c21-4ef13480f59a None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:36.356828 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:36.409210 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a9660-9d2f-4b70-9c21-4ef13480f59a None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:36.452822 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508221a-025e-4b58-9c55-7272008d9c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:36.453495 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508221a-025e-4b58-9c55-7272008d9c6b None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:36.453770 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508221a-025e-4b58-9c55-7272008d9c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:36.454045 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508221a-025e-4b58-9c55-7272008d9c6b None admin] PATH_INFO: `/v3/groups/685ef4c7d36e45e5961212c7d6e6c614` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:36.455206 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f508221a-025e-4b58-9c55-7272008d9c6b None admin] RBAC: Authorizing `identity:get_group(group_id=685ef4c7d36e45e5961212c7d6e6c614)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:36.456653 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f508221a-025e-4b58-9c55-7272008d9c6b None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:36.458579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 188/387] 10.222.0.26 () {56 vars in 1180 bytes} [Fri Sep 9 20:14:36 2022] GET /identity/v3/groups/685ef4c7d36e45e5961212c7d6e6c614 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:36.464638 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3706fa-b44c-4503-b839-298b7240b754 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:36.600598 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3706fa-b44c-4503-b839-298b7240b754 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:36.632199 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:36.645922 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c2e5d-c19c-4fa3-83de-2b0b1ec9f97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:36.646568 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c2e5d-c19c-4fa3-83de-2b0b1ec9f97d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:36.646848 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c2e5d-c19c-4fa3-83de-2b0b1ec9f97d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:36.647123 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c2e5d-c19c-4fa3-83de-2b0b1ec9f97d None admin] PATH_INFO: `/v3/projects/462d8ea458304464846aba18be1b8665` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:36.648317 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c2e5d-c19c-4fa3-83de-2b0b1ec9f97d None admin] RBAC: Authorizing `identity:get_project(project_id=462d8ea458304464846aba18be1b8665)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:36.649785 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c2e5d-c19c-4fa3-83de-2b0b1ec9f97d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:36.651421 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 200/388] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:14:36 2022] GET /identity/v3/projects/462d8ea458304464846aba18be1b8665 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:36.657288 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508221a-025e-4b58-9c55-7272008d9c6b None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:36.744133 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c2e5d-c19c-4fa3-83de-2b0b1ec9f97d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:36.786604 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508221a-025e-4b58-9c55-7272008d9c6b None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:36.822935 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a2da14-6e95-471d-850f-9608e45823e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:36.823613 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a2da14-6e95-471d-850f-9608e45823e5 None admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:36.823894 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a2da14-6e95-471d-850f-9608e45823e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:36.824169 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a2da14-6e95-471d-850f-9608e45823e5 None admin] PATH_INFO: `/v3/projects/462d8ea458304464846aba18be1b8665/groups/685ef4c7d36e45e5961212c7d6e6c614/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:36.826206 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a2da14-6e95-471d-850f-9608e45823e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=462d8ea458304464846aba18be1b8665, group_id=685ef4c7d36e45e5961212c7d6e6c614, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:36.827690 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a2da14-6e95-471d-850f-9608e45823e5 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:36.832313 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:36.853216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 189/389] 10.222.0.26 () {58 vars in 1442 bytes} [Fri Sep 9 20:14:36 2022] PUT /identity/v3/projects/462d8ea458304464846aba18be1b8665/groups/685ef4c7d36e45e5961212c7d6e6c614/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:14:36.926055 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c2e5d-c19c-4fa3-83de-2b0b1ec9f97d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:36.972508 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23500ba5-4484-4a49-89d5-9fe2d811c201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:36.973178 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23500ba5-4484-4a49-89d5-9fe2d811c201 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:36.973489 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23500ba5-4484-4a49-89d5-9fe2d811c201 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:36.973768 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23500ba5-4484-4a49-89d5-9fe2d811c201 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:36.991001 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23500ba5-4484-4a49-89d5-9fe2d811c201 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:36.998920 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-23500ba5-4484-4a49-89d5-9fe2d811c201 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:14:36.998920 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:36.998920 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:36.998920 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:36.998920 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:36.998920 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:36.998920 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:36.998920 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:36.998920 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:36.998920 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:36.998920 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:36.998920 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:36.998920 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:36.998920 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:36.998920 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:36.998920 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:37.000612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:37.000612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:37.000612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:37.000612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:37.000612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:37.000612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:37.000612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:37.000612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:37.000612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:37.000612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:37.000612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:37.000612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:37.000612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:37.000612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:37.000612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:37.000612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:37.001512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:37.001512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:37.001512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:37.001512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:37.001512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:37.001512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:37.001512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:37.001512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:37.001512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:37.001512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:37.001512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:14:37.001512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:37.001512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 201/390] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Sep 9 20:14:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:14:37.007407 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a2da14-6e95-471d-850f-9608e45823e5 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:37.010122 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc7f46-724b-4b5f-90f6-7b58e58d5164 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:37.010413 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc7f46-724b-4b5f-90f6-7b58e58d5164 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:37.010691 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc7f46-724b-4b5f-90f6-7b58e58d5164 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:37.012315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 202/391] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:14:37.017808 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9752b-c1b1-4031-a0be-350db5233afa None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:37.018095 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9752b-c1b1-4031-a0be-350db5233afa None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:37.018486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9752b-c1b1-4031-a0be-350db5233afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:37.048213 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-40d9752b-c1b1-4031-a0be-350db5233afa None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:37.050864 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-40d9752b-c1b1-4031-a0be-350db5233afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:37.157109 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:37.194053 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 203/392] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:37.217232 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a2da14-6e95-471d-850f-9608e45823e5 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:37.257115 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4040e1-a17f-4bfc-b54a-9c8233a2a1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:37.258629 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4040e1-a17f-4bfc-b54a-9c8233a2a1bd None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:37.258629 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4040e1-a17f-4bfc-b54a-9c8233a2a1bd None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:37.258629 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4040e1-a17f-4bfc-b54a-9c8233a2a1bd None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:37.259154 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4040e1-a17f-4bfc-b54a-9c8233a2a1bd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:37.259452 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4040e1-a17f-4bfc-b54a-9c8233a2a1bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:37.260805 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4040e1-a17f-4bfc-b54a-9c8233a2a1bd None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:37.270139 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 190/393] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:14:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:37.275606 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d9752b-c1b1-4031-a0be-350db5233afa None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:37.393079 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d9752b-c1b1-4031-a0be-350db5233afa None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:37.434573 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069637c-64eb-4549-96c5-5e174f13221c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:37.435215 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069637c-64eb-4549-96c5-5e174f13221c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:37.435493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069637c-64eb-4549-96c5-5e174f13221c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:37.435762 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069637c-64eb-4549-96c5-5e174f13221c None admin] PATH_INFO: `/v3/groups/685ef4c7d36e45e5961212c7d6e6c614` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:37.436873 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c069637c-64eb-4549-96c5-5e174f13221c None admin] RBAC: Authorizing `identity:get_group(group_id=685ef4c7d36e45e5961212c7d6e6c614)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:37.438329 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c069637c-64eb-4549-96c5-5e174f13221c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:37.440193 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 204/394] 10.222.0.26 () {56 vars in 1180 bytes} [Fri Sep 9 20:14:37 2022] GET /identity/v3/groups/685ef4c7d36e45e5961212c7d6e6c614 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:37.445576 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4040e1-a17f-4bfc-b54a-9c8233a2a1bd None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:37.511770 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:37.561233 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4040e1-a17f-4bfc-b54a-9c8233a2a1bd None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:37.572358 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069637c-64eb-4549-96c5-5e174f13221c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:37.607110 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652a693-58d0-4a98-97b6-d073a5d24dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:37.607747 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652a693-58d0-4a98-97b6-d073a5d24dfa None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:37.608021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652a693-58d0-4a98-97b6-d073a5d24dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:37.608290 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652a693-58d0-4a98-97b6-d073a5d24dfa None admin] PATH_INFO: `/v3/projects/462d8ea458304464846aba18be1b8665` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:37.609661 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0652a693-58d0-4a98-97b6-d073a5d24dfa None admin] RBAC: Authorizing `identity:get_project(project_id=462d8ea458304464846aba18be1b8665)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:37.611131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0652a693-58d0-4a98-97b6-d073a5d24dfa None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:37.612969 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 191/395] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:14:37 2022] GET /identity/v3/projects/462d8ea458304464846aba18be1b8665 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:37.618835 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652a693-58d0-4a98-97b6-d073a5d24dfa None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:37.649513 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:37.752086 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652a693-58d0-4a98-97b6-d073a5d24dfa None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:37.794168 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069637c-64eb-4549-96c5-5e174f13221c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:37.805399 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:37.831714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb6e93-5f5d-4311-a25d-3c2ae96bc665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:37.832343 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb6e93-5f5d-4311-a25d-3c2ae96bc665 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:37.832619 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb6e93-5f5d-4311-a25d-3c2ae96bc665 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:37.832892 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb6e93-5f5d-4311-a25d-3c2ae96bc665 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:37.833946 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e8f96a-6224-495d-b18c-96b0b54b9f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:37.834718 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e8f96a-6224-495d-b18c-96b0b54b9f58 None admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:37.835006 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e8f96a-6224-495d-b18c-96b0b54b9f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:37.835375 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e8f96a-6224-495d-b18c-96b0b54b9f58 None admin] PATH_INFO: `/v3/projects/462d8ea458304464846aba18be1b8665/groups/685ef4c7d36e45e5961212c7d6e6c614/roles/d5528405b19b48a6bc4b7ff451332d8a` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:37.841527 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e8f96a-6224-495d-b18c-96b0b54b9f58 None admin] RBAC: Authorizing `identity:create_grant(project_id=462d8ea458304464846aba18be1b8665, group_id=685ef4c7d36e45e5961212c7d6e6c614, role_id=d5528405b19b48a6bc4b7ff451332d8a)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:37.842985 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e8f96a-6224-495d-b18c-96b0b54b9f58 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:37.845829 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cb6e93-5f5d-4311-a25d-3c2ae96bc665 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:37.852493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-51cb6e93-5f5d-4311-a25d-3c2ae96bc665 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:14:37.852493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:37.852493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:37.852493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:37.852493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:37.852493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:37.852493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:37.852493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:37.852493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:37.852493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:37.852493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:37.852493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:37.852493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:37.852493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:37.852493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:37.852493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:37.853641 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:37.853641 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:37.853641 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:37.853641 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:37.853641 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:37.853641 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:37.853641 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:37.853641 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:37.853641 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:37.853641 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:37.853641 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:37.853641 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:37.853641 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:37.853641 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:37.853641 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:37.853641 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:37.854533 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:37.854533 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:37.854533 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:37.854533 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:37.854533 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:37.854533 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:37.854533 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:37.854533 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:37.854533 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:37.854533 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:37.854533 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:14:37.854533 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:37.854533 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 205/396] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:14:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:37.859921 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb6e93-5f5d-4311-a25d-3c2ae96bc665 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:37.875879 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 192/397] 10.222.0.26 () {58 vars in 1442 bytes} [Fri Sep 9 20:14:37 2022] PUT /identity/v3/projects/462d8ea458304464846aba18be1b8665/groups/685ef4c7d36e45e5961212c7d6e6c614/roles/d5528405b19b48a6bc4b7ff451332d8a => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:14:38.010984 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb6e93-5f5d-4311-a25d-3c2ae96bc665 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:38.030649 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:38.053738 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4bb22-7226-492a-a298-9223e0f3e88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:38.054391 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4bb22-7226-492a-a298-9223e0f3e88c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:38.054674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4bb22-7226-492a-a298-9223e0f3e88c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:38.054940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4bb22-7226-492a-a298-9223e0f3e88c None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:38.055630 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee4bb22-7226-492a-a298-9223e0f3e88c None admin] RBAC: Adding query filter params (name=member) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:38.055932 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee4bb22-7226-492a-a298-9223e0f3e88c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:38.057488 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee4bb22-7226-492a-a298-9223e0f3e88c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:38.067088 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 206/398] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Sep 9 20:14:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:38.073352 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e8f96a-6224-495d-b18c-96b0b54b9f58 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:38.202721 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e8f96a-6224-495d-b18c-96b0b54b9f58 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:38.240504 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901b802-e194-4749-9a30-66586c985542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:38.241121 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901b802-e194-4749-9a30-66586c985542 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:38.241420 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901b802-e194-4749-9a30-66586c985542 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:38.241709 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901b802-e194-4749-9a30-66586c985542 None admin] PATH_INFO: `/v3/groups/685ef4c7d36e45e5961212c7d6e6c614` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:38.242826 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b901b802-e194-4749-9a30-66586c985542 None admin] RBAC: Authorizing `identity:get_group(group_id=685ef4c7d36e45e5961212c7d6e6c614)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:38.244317 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b901b802-e194-4749-9a30-66586c985542 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:38.245991 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 193/399] 10.222.0.26 () {56 vars in 1180 bytes} [Fri Sep 9 20:14:38 2022] GET /identity/v3/groups/685ef4c7d36e45e5961212c7d6e6c614 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:38.251654 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4bb22-7226-492a-a298-9223e0f3e88c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:38.366082 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4bb22-7226-492a-a298-9223e0f3e88c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:38.402643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c38bb5-14e7-4d13-a8a5-b1ef95e8b0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:38.403280 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c38bb5-14e7-4d13-a8a5-b1ef95e8b0f1 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:38.403558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c38bb5-14e7-4d13-a8a5-b1ef95e8b0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:38.403832 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c38bb5-14e7-4d13-a8a5-b1ef95e8b0f1 None admin] PATH_INFO: `/v3/projects/445158de5da046e19c309424d250c953` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:38.405047 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c38bb5-14e7-4d13-a8a5-b1ef95e8b0f1 None admin] RBAC: Authorizing `identity:get_project(project_id=445158de5da046e19c309424d250c953)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:38.406530 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c38bb5-14e7-4d13-a8a5-b1ef95e8b0f1 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:38.408438 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 207/400] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:14:38 2022] GET /identity/v3/projects/445158de5da046e19c309424d250c953 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:38.414232 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901b802-e194-4749-9a30-66586c985542 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:38.474550 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb94f0f-9032-4328-83c9-97ed241165ae None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:38.474881 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb94f0f-9032-4328-83c9-97ed241165ae None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:38.475099 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb94f0f-9032-4328-83c9-97ed241165ae None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:38.476592 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 208/401] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:14:38.477654 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:38.481684 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e36ef1-998a-4b70-bdb6-f30ab15e63ba None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:38.482075 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e36ef1-998a-4b70-bdb6-f30ab15e63ba None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:38.482281 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e36ef1-998a-4b70-bdb6-f30ab15e63ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:38.509633 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-64e36ef1-998a-4b70-bdb6-f30ab15e63ba None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:38.511610 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-64e36ef1-998a-4b70-bdb6-f30ab15e63ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:38.574451 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901b802-e194-4749-9a30-66586c985542 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:38.600626 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:38.627638 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0aa4e7-8021-4a83-b378-d7ced1396384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:38.628230 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0aa4e7-8021-4a83-b378-d7ced1396384 None admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:38.628513 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0aa4e7-8021-4a83-b378-d7ced1396384 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:38.628789 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0aa4e7-8021-4a83-b378-d7ced1396384 None admin] PATH_INFO: `/v3/projects/445158de5da046e19c309424d250c953/groups/685ef4c7d36e45e5961212c7d6e6c614/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:38.631400 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0aa4e7-8021-4a83-b378-d7ced1396384 None admin] RBAC: Authorizing `identity:create_grant(project_id=445158de5da046e19c309424d250c953, group_id=685ef4c7d36e45e5961212c7d6e6c614, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:38.633443 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0aa4e7-8021-4a83-b378-d7ced1396384 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:38.651785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 194/402] 10.222.0.26 () {58 vars in 1442 bytes} [Fri Sep 9 20:14:38 2022] PUT /identity/v3/projects/445158de5da046e19c309424d250c953/groups/685ef4c7d36e45e5961212c7d6e6c614/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:14:38.691114 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 209/403] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:14:38.985346 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0aa4e7-8021-4a83-b378-d7ced1396384 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:38.991613 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae59a6-5cdd-4af0-86d6-43ce50e3b163 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:38.992011 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae59a6-5cdd-4af0-86d6-43ce50e3b163 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:38.992290 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae59a6-5cdd-4af0-86d6-43ce50e3b163 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:38.993967 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 210/404] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:39.006385 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde785ab-e974-4a32-987c-9fcc49d04996 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:39.006677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde785ab-e974-4a32-987c-9fcc49d04996 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:39.006994 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde785ab-e974-4a32-987c-9fcc49d04996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:39.028890 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-bde785ab-e974-4a32-987c-9fcc49d04996 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:39.030508 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-bde785ab-e974-4a32-987c-9fcc49d04996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:39.114139 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 211/405] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:39.169750 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0aa4e7-8021-4a83-b378-d7ced1396384 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:39.190470 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:39.218396 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7674313f-fd10-47f4-b121-60e928da8ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:39.219053 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674313f-fd10-47f4-b121-60e928da8ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:39.219331 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674313f-fd10-47f4-b121-60e928da8ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:39.219602 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674313f-fd10-47f4-b121-60e928da8ad0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:39.233967 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7674313f-fd10-47f4-b121-60e928da8ad0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:39.239308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-7674313f-fd10-47f4-b121-60e928da8ad0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:14:39.239308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:39.239308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:39.239308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:39.239308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:39.239308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:39.239308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:39.239308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:39.239308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:39.239308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:39.239308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:39.239308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:39.239308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:39.239308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:39.239308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:39.239308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:39.240162 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:39.240162 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:39.240162 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:39.240162 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:39.240162 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:39.240162 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:39.240162 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:39.240162 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:39.240162 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:39.240162 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:39.240162 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:39.240162 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:39.240162 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:39.240162 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:39.240162 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:39.240162 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:39.241458 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:39.241458 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:39.241458 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:39.241458 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:39.241458 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:39.241458 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:39.241458 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:39.241458 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:39.241458 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:39.241458 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:39.241458 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:14:39.241458 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:14:39.241458 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 195/406] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Sep 9 20:14:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:39.249114 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde785ab-e974-4a32-987c-9fcc49d04996 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:39.415938 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde785ab-e974-4a32-987c-9fcc49d04996 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:39.430304 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:39.494066 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd164e83-0b3f-49a4-ab3a-febdf6a63b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:39.495219 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd164e83-0b3f-49a4-ab3a-febdf6a63b81 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:39.495712 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd164e83-0b3f-49a4-ab3a-febdf6a63b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:39.496219 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd164e83-0b3f-49a4-ab3a-febdf6a63b81 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:39.497091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd164e83-0b3f-49a4-ab3a-febdf6a63b81 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:39.497607 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd164e83-0b3f-49a4-ab3a-febdf6a63b81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:39.499148 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd164e83-0b3f-49a4-ab3a-febdf6a63b81 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:39.511105 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 212/407] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:14:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:39.518315 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7674313f-fd10-47f4-b121-60e928da8ad0 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:39.610024 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd164e83-0b3f-49a4-ab3a-febdf6a63b81 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:39.690477 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7674313f-fd10-47f4-b121-60e928da8ad0 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:39.762814 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a764f-34ec-49a1-9a83-675c88696146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:39.763401 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a764f-34ec-49a1-9a83-675c88696146 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:39.763684 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a764f-34ec-49a1-9a83-675c88696146 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:39.764062 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a764f-34ec-49a1-9a83-675c88696146 None admin] PATH_INFO: `/v3/groups/685ef4c7d36e45e5961212c7d6e6c614` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:39.769080 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a764f-34ec-49a1-9a83-675c88696146 None admin] RBAC: Authorizing `identity:get_group(group_id=685ef4c7d36e45e5961212c7d6e6c614)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:39.770686 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a764f-34ec-49a1-9a83-675c88696146 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:39.771342 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd164e83-0b3f-49a4-ab3a-febdf6a63b81 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:39.772558 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 196/408] 10.222.0.26 () {56 vars in 1180 bytes} [Fri Sep 9 20:14:39 2022] GET /identity/v3/groups/685ef4c7d36e45e5961212c7d6e6c614 => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:39.786239 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a764f-34ec-49a1-9a83-675c88696146 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:39.813904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33c361-b00a-4832-8760-351151d6dee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:39.814535 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33c361-b00a-4832-8760-351151d6dee1 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:39.814814 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33c361-b00a-4832-8760-351151d6dee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:39.815085 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33c361-b00a-4832-8760-351151d6dee1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:39.834397 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a33c361-b00a-4832-8760-351151d6dee1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:39.839637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-4a33c361-b00a-4832-8760-351151d6dee1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:14:39.839637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:39.839637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:39.839637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:39.839637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:39.839637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:39.839637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:39.839637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:39.839637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:39.839637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:39.839637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:39.839637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:39.839637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:39.839637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:39.839637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:39.839637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:39.840569 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:39.840569 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:39.840569 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:39.840569 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:39.840569 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:39.840569 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:39.840569 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:39.840569 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:39.840569 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:39.840569 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:39.840569 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:39.840569 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:39.840569 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:39.840569 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:39.840569 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:39.840569 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:39.841405 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:39.841405 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:39.841405 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:39.841405 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:39.841405 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:39.841405 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:39.841405 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:39.841405 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:39.841405 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:39.841405 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:39.841405 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:14:39.841405 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:39.841405 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 213/409] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:14:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:14:39.846612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33c361-b00a-4832-8760-351151d6dee1 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:39.865716 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:39.873178 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:39.952446 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a764f-34ec-49a1-9a83-675c88696146 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:39.961441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33c361-b00a-4832-8760-351151d6dee1 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:39.997809 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb7757-deba-4ac8-af02-c068b711a320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:39.998770 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb7757-deba-4ac8-af02-c068b711a320 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:39.999364 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb7757-deba-4ac8-af02-c068b711a320 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:39.999787 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb7757-deba-4ac8-af02-c068b711a320 None admin] PATH_INFO: `/v3/projects/445158de5da046e19c309424d250c953` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:40.000467 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448fad6f-8281-4d40-a7f2-71287bbd1918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:40.001048 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448fad6f-8281-4d40-a7f2-71287bbd1918 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:40.001362 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448fad6f-8281-4d40-a7f2-71287bbd1918 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:40.001575 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eb7757-deba-4ac8-af02-c068b711a320 None admin] RBAC: Authorizing `identity:get_project(project_id=445158de5da046e19c309424d250c953)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:40.001694 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448fad6f-8281-4d40-a7f2-71287bbd1918 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:40.002299 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448fad6f-8281-4d40-a7f2-71287bbd1918 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:40.002602 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448fad6f-8281-4d40-a7f2-71287bbd1918 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:40.003517 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eb7757-deba-4ac8-af02-c068b711a320 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:40.003956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448fad6f-8281-4d40-a7f2-71287bbd1918 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:40.011248 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 197/410] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:14:39 2022] GET /identity/v3/projects/445158de5da046e19c309424d250c953 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:40.012647 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef15103-c2da-49b8-9db5-9cab53f87f46 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:40.013148 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef15103-c2da-49b8-9db5-9cab53f87f46 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:40.013581 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef15103-c2da-49b8-9db5-9cab53f87f46 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:40.013943 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 214/411] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:14:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:40.014732 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448fad6f-8281-4d40-a7f2-71287bbd1918 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:40.015636 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 198/412] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:40.021418 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8476b8-15bb-4c2f-ba5e-d05c3f7b950d None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:40.021952 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8476b8-15bb-4c2f-ba5e-d05c3f7b950d None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:40.022397 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8476b8-15bb-4c2f-ba5e-d05c3f7b950d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:40.042386 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-7f8476b8-15bb-4c2f-ba5e-d05c3f7b950d None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:40.043814 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7f8476b8-15bb-4c2f-ba5e-d05c3f7b950d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:40.126973 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:40.128169 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 199/413] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:40.129031 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8476b8-15bb-4c2f-ba5e-d05c3f7b950d None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:40.146658 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448fad6f-8281-4d40-a7f2-71287bbd1918 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:40.181266 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:40.188189 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ac5ac-2034-46df-bc12-f4fef3f42321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:40.188868 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ac5ac-2034-46df-bc12-f4fef3f42321 None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:40.189151 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ac5ac-2034-46df-bc12-f4fef3f42321 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:40.189454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ac5ac-2034-46df-bc12-f4fef3f42321 None admin] PATH_INFO: `/v3/projects/445158de5da046e19c309424d250c953/groups/685ef4c7d36e45e5961212c7d6e6c614/roles/d5528405b19b48a6bc4b7ff451332d8a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:40.192467 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ac5ac-2034-46df-bc12-f4fef3f42321 None admin] RBAC: Authorizing `identity:create_grant(project_id=445158de5da046e19c309424d250c953, group_id=685ef4c7d36e45e5961212c7d6e6c614, role_id=d5528405b19b48a6bc4b7ff451332d8a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:40.194097 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ac5ac-2034-46df-bc12-f4fef3f42321 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:40.211197 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 215/414] 10.222.0.26 () {58 vars in 1442 bytes} [Fri Sep 9 20:14:40 2022] PUT /identity/v3/projects/445158de5da046e19c309424d250c953/groups/685ef4c7d36e45e5961212c7d6e6c614/roles/d5528405b19b48a6bc4b7ff451332d8a => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:14:40.272495 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8476b8-15bb-4c2f-ba5e-d05c3f7b950d None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:40.322907 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4c2d8-5a17-4c3b-9a34-ec7a1b0dc0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:40.323846 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4c2d8-5a17-4c3b-9a34-ec7a1b0dc0a3 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:40.324293 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4c2d8-5a17-4c3b-9a34-ec7a1b0dc0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:40.324759 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4c2d8-5a17-4c3b-9a34-ec7a1b0dc0a3 None admin] PATH_INFO: `/v3/groups/e4c59e51c2e144c18fc58ca427948854` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:40.326115 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa4c2d8-5a17-4c3b-9a34-ec7a1b0dc0a3 None admin] RBAC: Authorizing `identity:get_group(group_id=e4c59e51c2e144c18fc58ca427948854)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:40.327715 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa4c2d8-5a17-4c3b-9a34-ec7a1b0dc0a3 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:40.330764 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 200/415] 10.222.0.26 () {56 vars in 1180 bytes} [Fri Sep 9 20:14:40 2022] GET /identity/v3/groups/e4c59e51c2e144c18fc58ca427948854 => generated 229 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:40.336745 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ac5ac-2034-46df-bc12-f4fef3f42321 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:40.457824 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ac5ac-2034-46df-bc12-f4fef3f42321 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:40.496818 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3fa37f-e5ae-4429-8845-4b03cf3f86cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:40.497804 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3fa37f-e5ae-4429-8845-4b03cf3f86cd None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:40.498256 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3fa37f-e5ae-4429-8845-4b03cf3f86cd None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:40.498696 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3fa37f-e5ae-4429-8845-4b03cf3f86cd None admin] PATH_INFO: `/v3/projects/e8a4d19e4885495c84cf29b80391fd52` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:40.500085 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3fa37f-e5ae-4429-8845-4b03cf3f86cd None admin] RBAC: Authorizing `identity:get_project(project_id=e8a4d19e4885495c84cf29b80391fd52)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:40.501730 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3fa37f-e5ae-4429-8845-4b03cf3f86cd None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:40.503912 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 216/416] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:14:40 2022] GET /identity/v3/projects/e8a4d19e4885495c84cf29b80391fd52 => generated 344 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:40.513780 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4c2d8-5a17-4c3b-9a34-ec7a1b0dc0a3 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:40.516676 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3fa37f-e5ae-4429-8845-4b03cf3f86cd None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:40.538122 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:40.645502 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4c2d8-5a17-4c3b-9a34-ec7a1b0dc0a3 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:40.651019 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3fa37f-e5ae-4429-8845-4b03cf3f86cd None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:40.657569 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:40.692943 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5252326f-36d0-4d81-a338-9dcace09fdbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:40.693647 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252326f-36d0-4d81-a338-9dcace09fdbd None admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:40.693923 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252326f-36d0-4d81-a338-9dcace09fdbd None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:40.694194 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252326f-36d0-4d81-a338-9dcace09fdbd None admin] PATH_INFO: `/v3/projects/e8a4d19e4885495c84cf29b80391fd52/groups/e4c59e51c2e144c18fc58ca427948854/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:40.696351 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5252326f-36d0-4d81-a338-9dcace09fdbd None admin] RBAC: Authorizing `identity:create_grant(project_id=e8a4d19e4885495c84cf29b80391fd52, group_id=e4c59e51c2e144c18fc58ca427948854, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:40.697209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cdb3d2-cb19-4669-85e4-75175aa69cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:40.697829 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5252326f-36d0-4d81-a338-9dcace09fdbd None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:40.698325 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdb3d2-cb19-4669-85e4-75175aa69cdd None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:40.698811 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdb3d2-cb19-4669-85e4-75175aa69cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:40.699276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdb3d2-cb19-4669-85e4-75175aa69cdd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:40.711760 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 201/417] 10.222.0.26 () {58 vars in 1442 bytes} [Fri Sep 9 20:14:40 2022] PUT /identity/v3/projects/e8a4d19e4885495c84cf29b80391fd52/groups/e4c59e51c2e144c18fc58ca427948854/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:14:40.712720 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c2010-1683-4970-9bc2-fbbb92d89a74 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:40.713067 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c2010-1683-4970-9bc2-fbbb92d89a74 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:40.713387 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c2010-1683-4970-9bc2-fbbb92d89a74 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:40.714092 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cdb3d2-cb19-4669-85e4-75175aa69cdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:40.714920 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 202/418] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:40.719891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-e0cdb3d2-cb19-4669-85e4-75175aa69cdd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:14:40.719891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:40.719891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:40.719891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:40.719891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:40.719891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:40.719891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:40.719891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:40.719891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:40.719891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:40.719891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:40.719891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:40.719891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:40.719891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:40.719891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:40.719891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:40.720890 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:40.720890 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:40.720890 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:40.720890 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:40.720890 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:40.720890 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:40.720890 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:40.720890 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:40.720890 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:40.720890 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:40.720890 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:40.720890 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:40.720890 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:40.720890 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:40.720890 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:40.720890 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:40.721735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:40.721735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:40.721735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:40.721735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:40.721735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:40.721735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:40.721735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:40.721735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:40.721735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:40.721735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:40.721735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:14:40.721735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:40.722396 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c78b8-310e-494f-bd89-6ad8335c6805 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:40.722396 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c78b8-310e-494f-bd89-6ad8335c6805 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:40.722396 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c78b8-310e-494f-bd89-6ad8335c6805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:40.730322 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 217/419] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:14:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:14:40.733057 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cdb3d2-cb19-4669-85e4-75175aa69cdd None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:40.741525 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-ae0c78b8-310e-494f-bd89-6ad8335c6805 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:40.742843 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-ae0c78b8-310e-494f-bd89-6ad8335c6805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:40.839564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:40.874322 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cdb3d2-cb19-4669-85e4-75175aa69cdd None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:40.879940 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 203/420] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:14:40.915104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b03e0f-3808-4832-8884-857c5177c648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:40.916104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b03e0f-3808-4832-8884-857c5177c648 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:40.916585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b03e0f-3808-4832-8884-857c5177c648 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:40.917082 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b03e0f-3808-4832-8884-857c5177c648 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:40.917984 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b03e0f-3808-4832-8884-857c5177c648 None admin] RBAC: Adding query filter params (name=member) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:40.918486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b03e0f-3808-4832-8884-857c5177c648 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:40.920062 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b03e0f-3808-4832-8884-857c5177c648 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:40.930048 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 218/421] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Sep 9 20:14:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:40.936250 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c78b8-310e-494f-bd89-6ad8335c6805 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:41.036145 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:41.042906 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c78b8-310e-494f-bd89-6ad8335c6805 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:41.086600 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46265ad-5327-470f-b25a-7a2167acacec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:41.087214 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46265ad-5327-470f-b25a-7a2167acacec None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:41.087550 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46265ad-5327-470f-b25a-7a2167acacec None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:41.087826 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46265ad-5327-470f-b25a-7a2167acacec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:41.173307 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46265ad-5327-470f-b25a-7a2167acacec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:41.174785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46265ad-5327-470f-b25a-7a2167acacec None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:41.215337 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b03e0f-3808-4832-8884-857c5177c648 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:41.331374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:41.345557 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 204/422] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:14:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:14:41.351631 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46265ad-5327-470f-b25a-7a2167acacec None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:41.418391 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b03e0f-3808-4832-8884-857c5177c648 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:41.502172 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f59bc-df56-40d3-b13a-6bc1a30a301d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:41.502809 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f59bc-df56-40d3-b13a-6bc1a30a301d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:41.503062 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f59bc-df56-40d3-b13a-6bc1a30a301d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:41.503333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f59bc-df56-40d3-b13a-6bc1a30a301d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:41.513158 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:41.524351 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6f59bc-df56-40d3-b13a-6bc1a30a301d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:41.536066 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-2d6f59bc-df56-40d3-b13a-6bc1a30a301d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:14:41.536066 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:41.536066 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:41.536066 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:41.536066 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:41.536066 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:41.536066 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:41.536066 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:41.536066 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:41.536066 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:41.536066 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:41.536066 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:41.536066 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:41.536066 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:41.536066 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:41.536066 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:41.537000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:41.537000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:41.537000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:41.537000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:41.537000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:41.537000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:41.537000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:41.537000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:41.537000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:41.537000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:41.537000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:41.537000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:41.537000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:41.537000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:41.537000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:41.537000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:41.537849 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:41.537849 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:41.537849 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:41.537849 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:41.537849 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:41.537849 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:41.537849 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:41.537849 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:41.537849 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:41.537849 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:41.537849 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:14:41.537849 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:41.538491 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 219/423] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Sep 9 20:14:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:41.544385 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f59bc-df56-40d3-b13a-6bc1a30a301d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:41.557372 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46265ad-5327-470f-b25a-7a2167acacec None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:41.597279 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0937c1b5-8d0d-4e6f-a9f1-43f5b063c9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:41.597901 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937c1b5-8d0d-4e6f-a9f1-43f5b063c9e7 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:41.598332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937c1b5-8d0d-4e6f-a9f1-43f5b063c9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:41.598607 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937c1b5-8d0d-4e6f-a9f1-43f5b063c9e7 None admin] PATH_INFO: `/v3/projects/462d8ea458304464846aba18be1b8665` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:41.600662 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0937c1b5-8d0d-4e6f-a9f1-43f5b063c9e7 None admin] RBAC: Authorizing `identity:get_project(project_id=462d8ea458304464846aba18be1b8665)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:41.602185 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0937c1b5-8d0d-4e6f-a9f1-43f5b063c9e7 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:41.603810 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 205/424] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:14:41 2022] GET /identity/v3/projects/462d8ea458304464846aba18be1b8665 => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:41.605410 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59914206-bd7c-42ce-bd96-182f96bc0010 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:41.605687 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59914206-bd7c-42ce-bd96-182f96bc0010 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:41.605956 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59914206-bd7c-42ce-bd96-182f96bc0010 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:41.607515 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 206/425] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:41.612210 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59914206-bd7c-42ce-bd96-182f96bc0010 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:41.678754 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f59bc-df56-40d3-b13a-6bc1a30a301d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:41.693599 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:41.719489 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148aad9-7ebb-4a4f-9951-1280ad17a561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:41.720365 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148aad9-7ebb-4a4f-9951-1280ad17a561 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:41.720840 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148aad9-7ebb-4a4f-9951-1280ad17a561 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:41.721205 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148aad9-7ebb-4a4f-9951-1280ad17a561 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:41.721992 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6148aad9-7ebb-4a4f-9951-1280ad17a561 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:41.722388 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6148aad9-7ebb-4a4f-9951-1280ad17a561 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:41.724029 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6148aad9-7ebb-4a4f-9951-1280ad17a561 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:41.731954 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59914206-bd7c-42ce-bd96-182f96bc0010 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:41.734176 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 220/426] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:14:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:41.735609 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f01796-1b42-4b33-9614-c9ef26017356 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:41.735989 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f01796-1b42-4b33-9614-c9ef26017356 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:41.736377 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f01796-1b42-4b33-9614-c9ef26017356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:41.756967 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e7f01796-1b42-4b33-9614-c9ef26017356 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:41.758295 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e7f01796-1b42-4b33-9614-c9ef26017356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:41.776378 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c4a6f-8f42-47b4-a173-8561bd5338f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:41.777323 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c4a6f-8f42-47b4-a173-8561bd5338f8 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:41.777610 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c4a6f-8f42-47b4-a173-8561bd5338f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:41.778255 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c4a6f-8f42-47b4-a173-8561bd5338f8 None admin] PATH_INFO: `/v3/groups/685ef4c7d36e45e5961212c7d6e6c614` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:41.780351 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c4a6f-8f42-47b4-a173-8561bd5338f8 None admin] RBAC: Authorizing `identity:get_group(group_id=685ef4c7d36e45e5961212c7d6e6c614)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:41.782076 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c4a6f-8f42-47b4-a173-8561bd5338f8 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:41.785325 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 207/427] 10.222.0.26 () {56 vars in 1180 bytes} [Fri Sep 9 20:14:41 2022] GET /identity/v3/groups/685ef4c7d36e45e5961212c7d6e6c614 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:41.786192 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c4a6f-8f42-47b4-a173-8561bd5338f8 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:41.820234 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:41.834897 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 221/428] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:41.835989 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f01796-1b42-4b33-9614-c9ef26017356 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:41.936202 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c4a6f-8f42-47b4-a173-8561bd5338f8 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:42.002935 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:42.005605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f01796-1b42-4b33-9614-c9ef26017356 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:42.012648 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed18a8b-e3aa-4c9c-9fb5-bea4331b293c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:42.014840 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed18a8b-e3aa-4c9c-9fb5-bea4331b293c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:42.015361 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed18a8b-e3aa-4c9c-9fb5-bea4331b293c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:42.017781 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed18a8b-e3aa-4c9c-9fb5-bea4331b293c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:42.075121 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c882006f-4dce-47b9-a83e-e885711401b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:42.077438 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882006f-4dce-47b9-a83e-e885711401b1 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:42.077917 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882006f-4dce-47b9-a83e-e885711401b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:42.078571 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882006f-4dce-47b9-a83e-e885711401b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:42.080540 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c882006f-4dce-47b9-a83e-e885711401b1 None admin] RBAC: Adding query filter params (group.id=685ef4c7d36e45e5961212c7d6e6c614, role.id=63fc659c11144d369a3f3fa684e056ae, scope.project.id=462d8ea458304464846aba18be1b8665) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:42.081810 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c882006f-4dce-47b9-a83e-e885711401b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:42.083360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c882006f-4dce-47b9-a83e-e885711401b1 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:42.093072 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 222/429] 10.222.0.26 () {56 vars in 1376 bytes} [Fri Sep 9 20:14:41 2022] GET /identity/v3/role_assignments?group.id=685ef4c7d36e45e5961212c7d6e6c614&scope.project.id=462d8ea458304464846aba18be1b8665&role.id=63fc659c11144d369a3f3fa684e056ae => generated 611 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:42.119984 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed18a8b-e3aa-4c9c-9fb5-bea4331b293c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:42.121942 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed18a8b-e3aa-4c9c-9fb5-bea4331b293c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:42.126143 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c882006f-4dce-47b9-a83e-e885711401b1 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:42.192175 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:42.249881 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c882006f-4dce-47b9-a83e-e885711401b1 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:42.259205 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 208/430] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:14:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:14:42.260731 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed18a8b-e3aa-4c9c-9fb5-bea4331b293c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:42.326417 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1a969f-f1dd-4d5a-9063-ddc516c44d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:42.326977 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a969f-f1dd-4d5a-9063-ddc516c44d48 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:42.327252 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a969f-f1dd-4d5a-9063-ddc516c44d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:42.327518 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a969f-f1dd-4d5a-9063-ddc516c44d48 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:42.331470 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:42.354081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1a969f-f1dd-4d5a-9063-ddc516c44d48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:42.359418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-fa1a969f-f1dd-4d5a-9063-ddc516c44d48 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:14:42.359418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:42.359418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:42.359418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:42.359418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:42.359418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:42.359418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:42.359418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:42.359418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:42.359418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:42.359418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:42.359418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:42.359418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:42.359418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:42.359418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:42.359418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:42.360376 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:42.360376 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:42.360376 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:42.360376 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:42.360376 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:42.360376 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:42.360376 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:42.360376 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:42.360376 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:42.360376 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:42.360376 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:42.360376 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:42.360376 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:42.360376 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:42.360376 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:42.360376 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:42.361209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:42.361209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:42.361209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:42.361209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:42.361209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:42.361209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:42.361209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:42.361209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:42.361209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:42.361209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:42.361209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:14:42.361209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:42.361856 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 223/431] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:14:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:42.367895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1a969f-f1dd-4d5a-9063-ddc516c44d48 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:42.438206 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed18a8b-e3aa-4c9c-9fb5-bea4331b293c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:42.492795 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1a969f-f1dd-4d5a-9063-ddc516c44d48 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:42.493671 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca0948-2b19-4dce-bf48-9e123d49c0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:42.494311 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca0948-2b19-4dce-bf48-9e123d49c0a4 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:42.494596 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca0948-2b19-4dce-bf48-9e123d49c0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:42.494868 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca0948-2b19-4dce-bf48-9e123d49c0a4 None admin] PATH_INFO: `/v3/projects/462d8ea458304464846aba18be1b8665` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:42.497205 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fca0948-2b19-4dce-bf48-9e123d49c0a4 None admin] RBAC: Authorizing `identity:get_project(project_id=462d8ea458304464846aba18be1b8665)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:42.498685 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fca0948-2b19-4dce-bf48-9e123d49c0a4 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:42.500681 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 209/432] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:14:42 2022] GET /identity/v3/projects/462d8ea458304464846aba18be1b8665 => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:42.509094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca0948-2b19-4dce-bf48-9e123d49c0a4 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:42.514299 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:42.539864 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0825fe-490e-4837-9ada-5aab6b3ce06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:42.540849 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0825fe-490e-4837-9ada-5aab6b3ce06e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:42.541380 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0825fe-490e-4837-9ada-5aab6b3ce06e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:42.541743 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0825fe-490e-4837-9ada-5aab6b3ce06e None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:42.542508 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0825fe-490e-4837-9ada-5aab6b3ce06e None admin] RBAC: Adding query filter params (name=member) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:42.543993 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0825fe-490e-4837-9ada-5aab6b3ce06e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:42.545480 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0825fe-490e-4837-9ada-5aab6b3ce06e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:42.554795 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 224/433] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Sep 9 20:14:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:42.560350 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0825fe-490e-4837-9ada-5aab6b3ce06e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:42.638405 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:42.642899 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca0948-2b19-4dce-bf48-9e123d49c0a4 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:42.691962 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0825fe-490e-4837-9ada-5aab6b3ce06e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:42.698618 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a838d-66d0-48a4-823f-0d8f169c335f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:42.699518 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a838d-66d0-48a4-823f-0d8f169c335f None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:42.700007 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a838d-66d0-48a4-823f-0d8f169c335f None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:42.701002 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a838d-66d0-48a4-823f-0d8f169c335f None admin] PATH_INFO: `/v3/groups/685ef4c7d36e45e5961212c7d6e6c614` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:42.705961 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232a838d-66d0-48a4-823f-0d8f169c335f None admin] RBAC: Authorizing `identity:get_group(group_id=685ef4c7d36e45e5961212c7d6e6c614)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:42.707440 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232a838d-66d0-48a4-823f-0d8f169c335f None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:42.709064 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 210/434] 10.222.0.26 () {56 vars in 1180 bytes} [Fri Sep 9 20:14:42 2022] GET /identity/v3/groups/685ef4c7d36e45e5961212c7d6e6c614 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:42.715462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a838d-66d0-48a4-823f-0d8f169c335f None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:42.748907 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8ed6b-0804-4b9b-bf0f-7038b384d00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:42.749870 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8ed6b-0804-4b9b-bf0f-7038b384d00f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:42.750288 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8ed6b-0804-4b9b-bf0f-7038b384d00f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:42.750701 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8ed6b-0804-4b9b-bf0f-7038b384d00f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:42.805974 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:42.843177 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed8ed6b-0804-4b9b-bf0f-7038b384d00f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:42.844795 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed8ed6b-0804-4b9b-bf0f-7038b384d00f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:42.860207 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a838d-66d0-48a4-823f-0d8f169c335f None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:42.903295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e181f981-818d-4b1a-bcc4-216193cc78da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:42.903833 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181f981-818d-4b1a-bcc4-216193cc78da None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:42.904111 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181f981-818d-4b1a-bcc4-216193cc78da None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:42.904380 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181f981-818d-4b1a-bcc4-216193cc78da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:42.905110 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e181f981-818d-4b1a-bcc4-216193cc78da None admin] RBAC: Adding query filter params (group.id=685ef4c7d36e45e5961212c7d6e6c614, role.id=d5528405b19b48a6bc4b7ff451332d8a, scope.project.id=462d8ea458304464846aba18be1b8665) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:42.905445 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e181f981-818d-4b1a-bcc4-216193cc78da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:42.906879 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e181f981-818d-4b1a-bcc4-216193cc78da None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:42.914213 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 211/435] 10.222.0.26 () {56 vars in 1376 bytes} [Fri Sep 9 20:14:42 2022] GET /identity/v3/role_assignments?group.id=685ef4c7d36e45e5961212c7d6e6c614&scope.project.id=462d8ea458304464846aba18be1b8665&role.id=d5528405b19b48a6bc4b7ff451332d8a => generated 611 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:42.915446 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657eb5d1-3c8c-4154-a00a-6711e5798bd6 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:42.915760 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657eb5d1-3c8c-4154-a00a-6711e5798bd6 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:42.916007 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657eb5d1-3c8c-4154-a00a-6711e5798bd6 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:42.917300 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 212/436] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:42.923586 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4491e0-0607-420d-9dde-a4ccef3da6b3 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:42.923917 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4491e0-0607-420d-9dde-a4ccef3da6b3 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:42.924140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4491e0-0607-420d-9dde-a4ccef3da6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:42.947228 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-1a4491e0-0607-420d-9dde-a4ccef3da6b3 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:42.948435 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-1a4491e0-0607-420d-9dde-a4ccef3da6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:42.989063 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 225/437] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:14:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 20:14:42.992441 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:42.998196 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8ed6b-0804-4b9b-bf0f-7038b384d00f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:43.061332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 213/438] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:14:43.115602 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:43.142665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8ed6b-0804-4b9b-bf0f-7038b384d00f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:43.177949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29466dc-d54e-4613-a149-3261e6625648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:43.178556 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29466dc-d54e-4613-a149-3261e6625648 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:43.178839 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29466dc-d54e-4613-a149-3261e6625648 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:43.179142 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29466dc-d54e-4613-a149-3261e6625648 None admin] PATH_INFO: `/v3/projects/445158de5da046e19c309424d250c953` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:43.180277 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29466dc-d54e-4613-a149-3261e6625648 None admin] RBAC: Authorizing `identity:get_project(project_id=445158de5da046e19c309424d250c953)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:43.181737 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29466dc-d54e-4613-a149-3261e6625648 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:43.183708 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 226/439] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:14:42 2022] GET /identity/v3/projects/445158de5da046e19c309424d250c953 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:43.188955 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4491e0-0607-420d-9dde-a4ccef3da6b3 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:43.309638 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4491e0-0607-420d-9dde-a4ccef3da6b3 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:43.348664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29466dc-d54e-4613-a149-3261e6625648 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:43.357050 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eae96c-b548-45a6-ae04-a41553312dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:43.357841 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eae96c-b548-45a6-ae04-a41553312dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:43.357841 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eae96c-b548-45a6-ae04-a41553312dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:43.358074 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eae96c-b548-45a6-ae04-a41553312dc7 None admin] PATH_INFO: `/v3/groups/685ef4c7d36e45e5961212c7d6e6c614` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:43.359025 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eae96c-b548-45a6-ae04-a41553312dc7 None admin] RBAC: Authorizing `identity:get_group(group_id=685ef4c7d36e45e5961212c7d6e6c614)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:43.361551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eae96c-b548-45a6-ae04-a41553312dc7 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:43.362877 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 214/440] 10.222.0.26 () {56 vars in 1180 bytes} [Fri Sep 9 20:14:43 2022] GET /identity/v3/groups/685ef4c7d36e45e5961212c7d6e6c614 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:43.372173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eae96c-b548-45a6-ae04-a41553312dc7 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:43.418806 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:43.479887 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29466dc-d54e-4613-a149-3261e6625648 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:43.498295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eae96c-b548-45a6-ae04-a41553312dc7 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:43.512973 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ffc67-537c-478d-bb66-114205dfd1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:43.513628 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ffc67-537c-478d-bb66-114205dfd1aa None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:43.513910 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ffc67-537c-478d-bb66-114205dfd1aa None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:43.514184 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ffc67-537c-478d-bb66-114205dfd1aa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:43.524876 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6ffc67-537c-478d-bb66-114205dfd1aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:43.530071 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-cb6ffc67-537c-478d-bb66-114205dfd1aa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:14:43.530071 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:43.530071 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:43.530071 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:43.530071 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:43.530071 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:43.530071 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:43.530071 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:43.530071 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:43.530071 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:43.530071 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:43.530071 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:43.530071 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:43.530071 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:43.530071 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:43.530071 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:43.530954 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:43.530954 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:43.530954 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:43.530954 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:43.530954 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:43.530954 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:43.530954 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:43.530954 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:43.530954 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:43.530954 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:43.530954 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:43.530954 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:43.530954 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:43.530954 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:43.530954 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:43.530954 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:43.531781 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:43.531781 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:43.531781 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:43.531781 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:43.531781 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:43.531781 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:43.531781 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:43.531781 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:43.531781 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:43.531781 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:43.531781 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:14:43.531781 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:43.531781 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 227/441] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Sep 9 20:14:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:43.532947 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f2460-3e68-43e9-ac5f-19095f92a394 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:43.533284 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f2460-3e68-43e9-ac5f-19095f92a394 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:43.533508 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f2460-3e68-43e9-ac5f-19095f92a394 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:43.534940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 228/442] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:43.535035 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d53976e-0e4d-430b-8b7d-b86135fa5106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:43.535560 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d53976e-0e4d-430b-8b7d-b86135fa5106 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:43.535880 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d53976e-0e4d-430b-8b7d-b86135fa5106 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:43.536422 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d53976e-0e4d-430b-8b7d-b86135fa5106 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:43.537116 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d53976e-0e4d-430b-8b7d-b86135fa5106 None admin] RBAC: Adding query filter params (group.id=685ef4c7d36e45e5961212c7d6e6c614, role.id=63fc659c11144d369a3f3fa684e056ae, scope.project.id=445158de5da046e19c309424d250c953) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:43.537461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d53976e-0e4d-430b-8b7d-b86135fa5106 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:43.538056 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f2460-3e68-43e9-ac5f-19095f92a394 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:43.539141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d53976e-0e4d-430b-8b7d-b86135fa5106 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:43.540919 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:43.545143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 215/443] 10.222.0.26 () {56 vars in 1376 bytes} [Fri Sep 9 20:14:43 2022] GET /identity/v3/role_assignments?group.id=685ef4c7d36e45e5961212c7d6e6c614&scope.project.id=445158de5da046e19c309424d250c953&role.id=63fc659c11144d369a3f3fa684e056ae => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:43.546337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991a032-a3c4-43d0-8d0c-dee3c80a390b None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:43.546670 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991a032-a3c4-43d0-8d0c-dee3c80a390b None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:43.546946 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991a032-a3c4-43d0-8d0c-dee3c80a390b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:43.566692 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-5991a032-a3c4-43d0-8d0c-dee3c80a390b None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:43.568101 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-5991a032-a3c4-43d0-8d0c-dee3c80a390b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:43.653271 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 216/444] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:43.663950 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f2460-3e68-43e9-ac5f-19095f92a394 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:43.706177 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371ea7f-4d5b-4d4a-a541-22de21dfd780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:43.706795 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371ea7f-4d5b-4d4a-a541-22de21dfd780 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:43.707078 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371ea7f-4d5b-4d4a-a541-22de21dfd780 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:43.707353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371ea7f-4d5b-4d4a-a541-22de21dfd780 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:43.708085 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e371ea7f-4d5b-4d4a-a541-22de21dfd780 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:43.708390 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e371ea7f-4d5b-4d4a-a541-22de21dfd780 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:43.709847 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e371ea7f-4d5b-4d4a-a541-22de21dfd780 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:43.719805 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 229/445] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:14:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:43.725935 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991a032-a3c4-43d0-8d0c-dee3c80a390b None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:43.756053 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:43.831726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991a032-a3c4-43d0-8d0c-dee3c80a390b None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:43.867645 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f60be-860a-439e-b1d1-f360c64afd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:43.868247 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f60be-860a-439e-b1d1-f360c64afd11 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:43.868522 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f60be-860a-439e-b1d1-f360c64afd11 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:43.868786 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f60be-860a-439e-b1d1-f360c64afd11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:43.915025 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371ea7f-4d5b-4d4a-a541-22de21dfd780 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:43.945805 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24f60be-860a-439e-b1d1-f360c64afd11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:43.946961 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24f60be-860a-439e-b1d1-f360c64afd11 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:43.974754 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:43.984533 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:44.023844 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371ea7f-4d5b-4d4a-a541-22de21dfd780 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:44.050007 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 217/446] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:14:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:14:44.056252 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f60be-860a-439e-b1d1-f360c64afd11 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:44.057647 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98fae2d-6a2f-4495-8c7f-3cbfb17015cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:44.058308 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98fae2d-6a2f-4495-8c7f-3cbfb17015cc None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:44.058591 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98fae2d-6a2f-4495-8c7f-3cbfb17015cc None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:44.058871 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98fae2d-6a2f-4495-8c7f-3cbfb17015cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:44.069363 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98fae2d-6a2f-4495-8c7f-3cbfb17015cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:44.074791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-d98fae2d-6a2f-4495-8c7f-3cbfb17015cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:14:44.074791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:44.074791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:44.074791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:44.074791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:44.074791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:44.074791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:44.074791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:44.074791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:44.074791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:44.074791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:44.074791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:44.074791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:44.074791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:44.074791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:44.074791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:44.075802 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:44.075802 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:44.075802 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:44.075802 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:44.075802 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:44.075802 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:44.075802 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:44.075802 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:44.075802 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:44.075802 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:44.075802 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:44.075802 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:44.075802 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:44.075802 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:44.075802 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:44.075802 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:44.076614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:44.076614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:44.076614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:44.076614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:44.076614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:44.076614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:44.076614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:44.076614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:44.076614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:44.076614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:44.076614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:14:44.076614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:44.076614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 230/447] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:14:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:14:44.081468 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98fae2d-6a2f-4495-8c7f-3cbfb17015cc None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:44.150757 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f60be-860a-439e-b1d1-f360c64afd11 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:44.173582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:44.182459 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98fae2d-6a2f-4495-8c7f-3cbfb17015cc None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:44.184633 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139044e1-0365-4130-8ab6-490947c9677b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:44.185271 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139044e1-0365-4130-8ab6-490947c9677b None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:44.185552 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139044e1-0365-4130-8ab6-490947c9677b None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:44.185823 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139044e1-0365-4130-8ab6-490947c9677b None admin] PATH_INFO: `/v3/projects/445158de5da046e19c309424d250c953` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:44.186921 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139044e1-0365-4130-8ab6-490947c9677b None admin] RBAC: Authorizing `identity:get_project(project_id=445158de5da046e19c309424d250c953)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:44.188344 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139044e1-0365-4130-8ab6-490947c9677b None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:44.189972 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 218/448] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:14:44 2022] GET /identity/v3/projects/445158de5da046e19c309424d250c953 => generated 302 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:44.195308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139044e1-0365-4130-8ab6-490947c9677b None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:44.205085 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:44.215613 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b67752-ae1f-4757-bfd0-7e3fa8a4d555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:44.216211 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b67752-ae1f-4757-bfd0-7e3fa8a4d555 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:44.216493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b67752-ae1f-4757-bfd0-7e3fa8a4d555 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:44.216764 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b67752-ae1f-4757-bfd0-7e3fa8a4d555 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:44.217445 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b67752-ae1f-4757-bfd0-7e3fa8a4d555 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:44.217746 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b67752-ae1f-4757-bfd0-7e3fa8a4d555 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:44.219100 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b67752-ae1f-4757-bfd0-7e3fa8a4d555 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:44.227983 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 231/449] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:14:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:44.233443 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b67752-ae1f-4757-bfd0-7e3fa8a4d555 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:44.295427 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139044e1-0365-4130-8ab6-490947c9677b None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:44.328092 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df00c9c-b8d7-4c22-b043-b4620038a8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:44.328716 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df00c9c-b8d7-4c22-b043-b4620038a8de None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:44.328994 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df00c9c-b8d7-4c22-b043-b4620038a8de None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:44.329287 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df00c9c-b8d7-4c22-b043-b4620038a8de None admin] PATH_INFO: `/v3/groups/685ef4c7d36e45e5961212c7d6e6c614` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:44.329594 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b67752-ae1f-4757-bfd0-7e3fa8a4d555 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:44.330372 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df00c9c-b8d7-4c22-b043-b4620038a8de None admin] RBAC: Authorizing `identity:get_group(group_id=685ef4c7d36e45e5961212c7d6e6c614)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:44.331828 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df00c9c-b8d7-4c22-b043-b4620038a8de None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:44.333398 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 219/450] 10.222.0.26 () {56 vars in 1180 bytes} [Fri Sep 9 20:14:44 2022] GET /identity/v3/groups/685ef4c7d36e45e5961212c7d6e6c614 => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:44.339632 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df00c9c-b8d7-4c22-b043-b4620038a8de None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:44.365998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17e587-a37e-4e80-9a1a-e4166c48047b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:44.366621 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17e587-a37e-4e80-9a1a-e4166c48047b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:44.366900 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17e587-a37e-4e80-9a1a-e4166c48047b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:44.367168 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17e587-a37e-4e80-9a1a-e4166c48047b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:44.398612 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:44.421811 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:44.435548 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a17e587-a37e-4e80-9a1a-e4166c48047b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:44.437001 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a17e587-a37e-4e80-9a1a-e4166c48047b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:44.439958 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df00c9c-b8d7-4c22-b043-b4620038a8de None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:44.473444 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d62798-e09b-4034-ab88-129292ba9758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:44.474049 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d62798-e09b-4034-ab88-129292ba9758 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:44.474326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d62798-e09b-4034-ab88-129292ba9758 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:44.474595 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d62798-e09b-4034-ab88-129292ba9758 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:44.475309 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d62798-e09b-4034-ab88-129292ba9758 None admin] RBAC: Adding query filter params (group.id=685ef4c7d36e45e5961212c7d6e6c614, role.id=d5528405b19b48a6bc4b7ff451332d8a, scope.project.id=445158de5da046e19c309424d250c953) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:44.475617 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d62798-e09b-4034-ab88-129292ba9758 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:44.477612 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d62798-e09b-4034-ab88-129292ba9758 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:44.483165 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 220/451] 10.222.0.26 () {56 vars in 1376 bytes} [Fri Sep 9 20:14:44 2022] GET /identity/v3/role_assignments?group.id=685ef4c7d36e45e5961212c7d6e6c614&scope.project.id=445158de5da046e19c309424d250c953&role.id=d5528405b19b48a6bc4b7ff451332d8a => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:44.537142 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 232/452] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:14:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 20:14:44.543713 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d62798-e09b-4034-ab88-129292ba9758 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:44.648621 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d62798-e09b-4034-ab88-129292ba9758 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:44.678016 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:44.689644 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1237f4-572a-4da9-907d-d5edb3999b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:44.690278 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1237f4-572a-4da9-907d-d5edb3999b8c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:44.690558 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1237f4-572a-4da9-907d-d5edb3999b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:44.690843 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1237f4-572a-4da9-907d-d5edb3999b8c None admin] PATH_INFO: `/v3/projects/e8a4d19e4885495c84cf29b80391fd52` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:44.691961 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1237f4-572a-4da9-907d-d5edb3999b8c None admin] RBAC: Authorizing `identity:get_project(project_id=e8a4d19e4885495c84cf29b80391fd52)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:44.693485 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1237f4-572a-4da9-907d-d5edb3999b8c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:44.695400 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 221/453] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:14:44 2022] GET /identity/v3/projects/e8a4d19e4885495c84cf29b80391fd52 => generated 344 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:44.700639 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17e587-a37e-4e80-9a1a-e4166c48047b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:44.788049 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:44.806276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17e587-a37e-4e80-9a1a-e4166c48047b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:44.839425 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0ce93-42e8-456a-b365-d5605a89b4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:44.840058 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0ce93-42e8-456a-b365-d5605a89b4b6 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:44.840335 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0ce93-42e8-456a-b365-d5605a89b4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:44.840605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0ce93-42e8-456a-b365-d5605a89b4b6 None admin] PATH_INFO: `/v3/groups/e4c59e51c2e144c18fc58ca427948854` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:44.841725 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c0ce93-42e8-456a-b365-d5605a89b4b6 None admin] RBAC: Authorizing `identity:get_group(group_id=e4c59e51c2e144c18fc58ca427948854)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:44.843148 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c0ce93-42e8-456a-b365-d5605a89b4b6 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:44.844867 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 233/454] 10.222.0.26 () {56 vars in 1180 bytes} [Fri Sep 9 20:14:44 2022] GET /identity/v3/groups/e4c59e51c2e144c18fc58ca427948854 => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:44.850550 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1237f4-572a-4da9-907d-d5edb3999b8c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:44.948362 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1237f4-572a-4da9-907d-d5edb3999b8c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:44.980454 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8be3a-38e6-4f78-a686-ea8615c0bf5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:44.981044 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8be3a-38e6-4f78-a686-ea8615c0bf5c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:44.981354 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8be3a-38e6-4f78-a686-ea8615c0bf5c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:44.981629 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8be3a-38e6-4f78-a686-ea8615c0bf5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:44.982346 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e8be3a-38e6-4f78-a686-ea8615c0bf5c None admin] RBAC: Adding query filter params (group.id=e4c59e51c2e144c18fc58ca427948854, role.id=9121ba25a6d84756990f0c3a19e9fecb, scope.project.id=e8a4d19e4885495c84cf29b80391fd52) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:44.982651 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e8be3a-38e6-4f78-a686-ea8615c0bf5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:44.984069 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e8be3a-38e6-4f78-a686-ea8615c0bf5c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:44.989630 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 222/455] 10.222.0.26 () {56 vars in 1376 bytes} [Fri Sep 9 20:14:44 2022] GET /identity/v3/role_assignments?group.id=e4c59e51c2e144c18fc58ca427948854&scope.project.id=e8a4d19e4885495c84cf29b80391fd52&role.id=9121ba25a6d84756990f0c3a19e9fecb => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:47.688778 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af8281-81cc-4803-bae8-6c8b79ed2984 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:47.689831 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af8281-81cc-4803-bae8-6c8b79ed2984 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:47.690310 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af8281-81cc-4803-bae8-6c8b79ed2984 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:47.692090 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 234/456] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:47.697845 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7618325a-afd1-4d5a-bd8d-52c009145a0c None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:47.698652 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7618325a-afd1-4d5a-bd8d-52c009145a0c None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:47.699206 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7618325a-afd1-4d5a-bd8d-52c009145a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:47.730234 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-7618325a-afd1-4d5a-bd8d-52c009145a0c None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:47.731785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7618325a-afd1-4d5a-bd8d-52c009145a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:47.799906 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:47.823721 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 223/457] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:14:48.039410 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1b7eb-7d7e-439f-8211-c4581225d38c None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:48.040116 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1b7eb-7d7e-439f-8211-c4581225d38c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:48.040603 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1b7eb-7d7e-439f-8211-c4581225d38c None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:48.041193 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f1d91-9fd5-4a14-a351-b31c84fd1ef8 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:48.041508 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f1d91-9fd5-4a14-a351-b31c84fd1ef8 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:48.041779 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f1d91-9fd5-4a14-a351-b31c84fd1ef8 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:48.043049 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 235/458] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:48.044281 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 224/459] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:48.048359 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef38d18-5b95-41b7-8639-0f7f9ea1663b None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:48.048879 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef38d18-5b95-41b7-8639-0f7f9ea1663b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:48.049392 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef38d18-5b95-41b7-8639-0f7f9ea1663b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:48.056106 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810b70f-e5ed-4f56-bb02-0e7260079d34 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:48.056619 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810b70f-e5ed-4f56-bb02-0e7260079d34 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:48.057088 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810b70f-e5ed-4f56-bb02-0e7260079d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:48.077402 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-7ef38d18-5b95-41b7-8639-0f7f9ea1663b None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:48.078592 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7ef38d18-5b95-41b7-8639-0f7f9ea1663b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:48.084448 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-c810b70f-e5ed-4f56-bb02-0e7260079d34 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:48.086162 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-c810b70f-e5ed-4f56-bb02-0e7260079d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:48.208764 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 236/460] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:14:48.209900 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 225/461] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:14:48.210841 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef38d18-5b95-41b7-8639-0f7f9ea1663b None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:48.301521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:48.327711 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef38d18-5b95-41b7-8639-0f7f9ea1663b None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:48.371599 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1409dfc-106b-47aa-b0c6-42d16f32a914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:48.372386 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1409dfc-106b-47aa-b0c6-42d16f32a914 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:48.372662 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1409dfc-106b-47aa-b0c6-42d16f32a914 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:48.372929 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1409dfc-106b-47aa-b0c6-42d16f32a914 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:48.379799 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f78154-6868-46ae-8602-398d655cccab None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:48.380536 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f78154-6868-46ae-8602-398d655cccab None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:48.381049 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f78154-6868-46ae-8602-398d655cccab None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:48.383493 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 226/462] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:14:48.388926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c29cd8-6714-4691-9ce6-133da5a6207a None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:48.389219 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c29cd8-6714-4691-9ce6-133da5a6207a None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:48.389577 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c29cd8-6714-4691-9ce6-133da5a6207a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:48.391909 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1409dfc-106b-47aa-b0c6-42d16f32a914 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:48.393424 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1409dfc-106b-47aa-b0c6-42d16f32a914 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:48.409353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-f1409dfc-106b-47aa-b0c6-42d16f32a914 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:14:48.409353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:48.409353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:48.409353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:48.409353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:48.409353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:48.409353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:48.409353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:48.409353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:48.409353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:48.409353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:48.409353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:48.409353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:14:48.409353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:14:48.409353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:14:48.409353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:14:48.410386 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:48.410386 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:48.410386 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:48.410386 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:48.410386 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:48.410386 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:48.410386 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:48.410386 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:48.410386 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:48.410386 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:48.410386 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:48.410386 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:48.410386 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:48.410386 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:48.410386 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:48.410386 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:48.410386 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:14:48.411240 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:14:48.411240 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:14:48.411240 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:48.413039 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 237/463] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:14:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:48.422190 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1409dfc-106b-47aa-b0c6-42d16f32a914 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:48.426868 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-21c29cd8-6714-4691-9ce6-133da5a6207a None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:48.428048 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-21c29cd8-6714-4691-9ce6-133da5a6207a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:48.540177 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1409dfc-106b-47aa-b0c6-42d16f32a914 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:48.561110 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 227/464] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:48.562308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c29cd8-6714-4691-9ce6-133da5a6207a None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:48.589832 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e485c9-f29c-4a76-8d37-beee8caa4b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:48.590491 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e485c9-f29c-4a76-8d37-beee8caa4b06 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:48.590770 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e485c9-f29c-4a76-8d37-beee8caa4b06 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:48.591042 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e485c9-f29c-4a76-8d37-beee8caa4b06 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:48.592296 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e485c9-f29c-4a76-8d37-beee8caa4b06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:48.592598 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e485c9-f29c-4a76-8d37-beee8caa4b06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:48.594453 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e485c9-f29c-4a76-8d37-beee8caa4b06 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:48.597277 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:48.612865 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 238/465] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:48.613893 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e485c9-f29c-4a76-8d37-beee8caa4b06 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:48.625704 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:48.698267 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c29cd8-6714-4691-9ce6-133da5a6207a None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:48.747560 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e485c9-f29c-4a76-8d37-beee8caa4b06 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:48.752883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d0b57e-ab46-4160-8a13-73c30e8448f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:48.753558 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d0b57e-ab46-4160-8a13-73c30e8448f6 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:48.753833 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d0b57e-ab46-4160-8a13-73c30e8448f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:48.754366 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d0b57e-ab46-4160-8a13-73c30e8448f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:48.768770 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d0b57e-ab46-4160-8a13-73c30e8448f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:48.770777 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d0b57e-ab46-4160-8a13-73c30e8448f6 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:48.788376 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cf8e65-03c8-4b27-ab47-cb3ed15725a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:48.789292 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf8e65-03c8-4b27-ab47-cb3ed15725a6 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:48.789765 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf8e65-03c8-4b27-ab47-cb3ed15725a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:48.790203 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf8e65-03c8-4b27-ab47-cb3ed15725a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:48.790456 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-11d0b57e-ab46-4160-8a13-73c30e8448f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:14:48.790456 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:48.790456 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:48.790456 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:48.790456 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:48.790456 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:48.790456 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:48.790456 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:48.790456 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:48.790456 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:48.790456 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:48.790456 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:48.790456 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:14:48.790456 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:14:48.790456 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:14:48.790456 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:14:48.791309 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:48.791309 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:48.791309 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:48.791309 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:48.791309 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:48.791309 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:48.791309 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:48.791309 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:48.791309 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:48.791309 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:48.791309 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:48.791309 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:48.791309 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:48.791309 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:48.791309 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:48.791309 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:48.791309 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:14:48.797170 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:14:48.797170 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:14:48.797170 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:14:48.797170 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 228/466] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:14:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:48.797170 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d0b57e-ab46-4160-8a13-73c30e8448f6 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:48.810020 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cf8e65-03c8-4b27-ab47-cb3ed15725a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:48.811872 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cf8e65-03c8-4b27-ab47-cb3ed15725a6 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:48.828581 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-55cf8e65-03c8-4b27-ab47-cb3ed15725a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:14:48.828581 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:48.828581 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:48.828581 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:48.828581 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:48.828581 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:48.828581 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:48.828581 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:48.828581 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:48.828581 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:48.828581 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:48.828581 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:48.828581 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:14:48.828581 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:14:48.828581 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:14:48.828581 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:14:48.829605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:48.829605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:48.829605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:48.829605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:48.829605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:48.829605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:48.829605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:48.829605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:48.829605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:48.829605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:48.829605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:48.829605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:48.829605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:48.829605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:48.829605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:48.829605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:48.829605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:14:48.830493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:14:48.830493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:14:48.830493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:48.837648 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 239/467] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:14:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:48.838827 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cf8e65-03c8-4b27-ab47-cb3ed15725a6 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:48.893185 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:48.906846 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:48.910939 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d0b57e-ab46-4160-8a13-73c30e8448f6 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:48.949389 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7aedd7-465f-4be7-b93d-5f92ba8f563e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:48.950010 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7aedd7-465f-4be7-b93d-5f92ba8f563e None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:48.950291 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7aedd7-465f-4be7-b93d-5f92ba8f563e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:48.950578 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7aedd7-465f-4be7-b93d-5f92ba8f563e None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:48.951212 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7aedd7-465f-4be7-b93d-5f92ba8f563e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:48.952646 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7aedd7-465f-4be7-b93d-5f92ba8f563e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:48.953095 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cf8e65-03c8-4b27-ab47-cb3ed15725a6 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:48.973198 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 229/468] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Sep 9 20:14:48 2022] POST /identity/v3/users => generated 246 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:14:48.974079 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7aedd7-465f-4be7-b93d-5f92ba8f563e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:49.000601 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8dd7a2-c998-4cdd-80a2-a67b20718f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:49.001939 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8dd7a2-c998-4cdd-80a2-a67b20718f7b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:49.002620 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8dd7a2-c998-4cdd-80a2-a67b20718f7b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:49.002899 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8dd7a2-c998-4cdd-80a2-a67b20718f7b None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:49.003595 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8dd7a2-c998-4cdd-80a2-a67b20718f7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:49.003900 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8dd7a2-c998-4cdd-80a2-a67b20718f7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:49.005663 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8dd7a2-c998-4cdd-80a2-a67b20718f7b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:49.021898 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 240/469] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:49.023603 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8dd7a2-c998-4cdd-80a2-a67b20718f7b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:49.154964 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8dd7a2-c998-4cdd-80a2-a67b20718f7b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:49.155856 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7aedd7-465f-4be7-b93d-5f92ba8f563e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:49.168311 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:49.192915 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae6fed-0eee-4fa5-892e-e6db08847551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:49.194447 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae6fed-0eee-4fa5-892e-e6db08847551 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:49.195328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae6fed-0eee-4fa5-892e-e6db08847551 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:49.195600 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae6fed-0eee-4fa5-892e-e6db08847551 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:49.196255 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ae6fed-0eee-4fa5-892e-e6db08847551 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:49.196556 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ae6fed-0eee-4fa5-892e-e6db08847551 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:49.198556 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ae6fed-0eee-4fa5-892e-e6db08847551 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:49.203506 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805be03e-1012-4280-985a-c2b02aa6f04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:49.204147 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805be03e-1012-4280-985a-c2b02aa6f04b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:49.204745 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805be03e-1012-4280-985a-c2b02aa6f04b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:49.204745 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805be03e-1012-4280-985a-c2b02aa6f04b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:49.212984 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 230/470] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:49.213663 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae6fed-0eee-4fa5-892e-e6db08847551 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:49.219751 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805be03e-1012-4280-985a-c2b02aa6f04b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:49.221868 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805be03e-1012-4280-985a-c2b02aa6f04b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:49.230981 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:49.243225 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-805be03e-1012-4280-985a-c2b02aa6f04b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:14:49.243225 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:49.243225 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:49.243225 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:49.243225 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:49.243225 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:49.243225 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:49.243225 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:49.243225 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:49.243225 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:49.243225 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:49.243225 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:49.243225 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:14:49.243225 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:14:49.243225 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:14:49.243225 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:14:49.244180 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:49.244180 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:49.244180 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:49.244180 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:49.244180 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:49.244180 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:49.244180 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:49.244180 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:49.244180 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:49.244180 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:49.244180 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:49.244180 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:49.244180 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:49.244180 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:49.244180 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:49.244180 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:49.244180 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:14:49.245036 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:14:49.245036 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:14:49.245036 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:49.245210 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 241/471] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:14:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:49.247068 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3b0bc-d521-45e6-b2af-07af82ddfa67 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:49.247068 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3b0bc-d521-45e6-b2af-07af82ddfa67 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:49.247068 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3b0bc-d521-45e6-b2af-07af82ddfa67 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:49.248352 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 242/472] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:49.249153 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3b0bc-d521-45e6-b2af-07af82ddfa67 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:49.337542 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae6fed-0eee-4fa5-892e-e6db08847551 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:49.364316 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3b0bc-d521-45e6-b2af-07af82ddfa67 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:49.372666 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6f76bf-ec0f-4636-b912-ab38559e820e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:49.373013 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f76bf-ec0f-4636-b912-ab38559e820e None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:49.373390 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f76bf-ec0f-4636-b912-ab38559e820e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:49.373703 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f76bf-ec0f-4636-b912-ab38559e820e None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:49.374408 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6f76bf-ec0f-4636-b912-ab38559e820e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:49.375824 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6f76bf-ec0f-4636-b912-ab38559e820e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:49.405369 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 231/473] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Sep 9 20:14:49 2022] POST /identity/v3/users => generated 248 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:14:49.405369 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6f76bf-ec0f-4636-b912-ab38559e820e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:49.417486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe5465-36d8-4536-8122-58404d480dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:49.418167 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe5465-36d8-4536-8122-58404d480dbc None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:49.418447 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe5465-36d8-4536-8122-58404d480dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:49.418719 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe5465-36d8-4536-8122-58404d480dbc None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:49.419400 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe5465-36d8-4536-8122-58404d480dbc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:49.421681 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe5465-36d8-4536-8122-58404d480dbc None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:49.455758 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 243/474] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Sep 9 20:14:49 2022] POST /identity/v3/users => generated 248 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:14:49.457262 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f93e5c-4a2f-4481-8541-85172d194997 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:49.457558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f93e5c-4a2f-4481-8541-85172d194997 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:49.457831 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f93e5c-4a2f-4481-8541-85172d194997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:49.482795 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-03f93e5c-4a2f-4481-8541-85172d194997 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:49.484454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-03f93e5c-4a2f-4481-8541-85172d194997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:49.503581 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:49.504472 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:49.545105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6f76bf-ec0f-4636-b912-ab38559e820e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:49.589100 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde5a800-ff31-4c4d-a9cb-0af7231e0705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:49.589775 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5a800-ff31-4c4d-a9cb-0af7231e0705 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:49.590035 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 244/475] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:49.590140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5a800-ff31-4c4d-a9cb-0af7231e0705 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:49.590358 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5a800-ff31-4c4d-a9cb-0af7231e0705 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:49.591017 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde5a800-ff31-4c4d-a9cb-0af7231e0705 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:49.591312 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde5a800-ff31-4c4d-a9cb-0af7231e0705 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:49.592711 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde5a800-ff31-4c4d-a9cb-0af7231e0705 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:49.608546 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 232/476] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:49.614620 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f93e5c-4a2f-4481-8541-85172d194997 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:49.736311 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f93e5c-4a2f-4481-8541-85172d194997 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:49.773670 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2421be-da92-4780-994e-fa8b63108640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:49.774302 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2421be-da92-4780-994e-fa8b63108640 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:49.774548 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2421be-da92-4780-994e-fa8b63108640 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:49.774815 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2421be-da92-4780-994e-fa8b63108640 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:49.775464 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2421be-da92-4780-994e-fa8b63108640 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:49.776892 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2421be-da92-4780-994e-fa8b63108640 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:49.799059 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 245/477] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Sep 9 20:14:49 2022] POST /identity/v3/users => generated 249 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:14:49.890700 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde5a800-ff31-4c4d-a9cb-0af7231e0705 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:50.008626 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde5a800-ff31-4c4d-a9cb-0af7231e0705 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:50.039222 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:50.045794 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c375cc88-565d-4810-b2d5-6cd3c4558433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:50.046411 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375cc88-565d-4810-b2d5-6cd3c4558433 None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:50.047044 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375cc88-565d-4810-b2d5-6cd3c4558433 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:50.047721 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375cc88-565d-4810-b2d5-6cd3c4558433 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:50.048399 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c375cc88-565d-4810-b2d5-6cd3c4558433 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:50.049828 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c375cc88-565d-4810-b2d5-6cd3c4558433 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:50.059125 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.sql.core [None req-c375cc88-565d-4810-b2d5-6cd3c4558433 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 20:14:50.059125 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 09 20:14:50.059125 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [parameters: {'id': '3e9f2d8c3c8e44469eb935681a990867', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 09 20:14:50.059125 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81430) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 20:14:50.076014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-c375cc88-565d-4810-b2d5-6cd3c4558433 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 09 20:14:50.076014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:50.076014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 20:14:50.076014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 20:14:50.076014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 20:14:50.076014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 20:14:50.076014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 20:14:50.076014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 20:14:50.076014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 20:14:50.076014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application conn.query(q) Sep 09 20:14:50.076014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 20:14:50.076014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 20:14:50.076014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 20:14:50.076014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application result.read() Sep 09 20:14:50.076014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 20:14:50.076926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 20:14:50.076926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 20:14:50.076926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 20:14:50.076926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 20:14:50.076926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 20:14:50.076926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 20:14:50.076926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 20:14:50.076926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 20:14:50.076926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:14:50.076926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 20:14:50.076926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:14:50.076926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:50.076926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 20:14:50.076926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 20:14:50.076926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 09 20:14:50.076926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return ref.to_dict() Sep 09 20:14:50.076926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 20:14:50.076926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application next(self.gen) Sep 09 20:14:50.077926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 20:14:50.077926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application yield resource Sep 09 20:14:50.077926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 20:14:50.077926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application next(self.gen) Sep 09 20:14:50.077926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 20:14:50.077926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 20:14:50.077926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 20:14:50.077926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 20:14:50.077926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 20:14:50.077926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise self.value Sep 09 20:14:50.077926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 20:14:50.077926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 20:14:50.077926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 20:14:50.077926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application session.commit() Sep 09 20:14:50.077926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 20:14:50.077926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 20:14:50.077926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 20:14:50.078796 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 20:14:50.078796 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 20:14:50.078796 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self.session.flush() Sep 09 20:14:50.078796 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 20:14:50.078796 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 20:14:50.078796 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 20:14:50.078796 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 20:14:50.078796 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 20:14:50.078796 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application compat.raise_( Sep 09 20:14:50.078796 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 20:14:50.078796 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception Sep 09 20:14:50.078796 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 20:14:50.078796 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 20:14:50.078796 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 20:14:50.078796 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 20:14:50.078796 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 20:14:50.078796 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 20:14:50.079791 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 20:14:50.079791 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 20:14:50.079791 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 09 20:14:50.079791 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 09 20:14:50.079791 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 20:14:50.079791 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 20:14:50.079791 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 20:14:50.079791 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 20:14:50.079791 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 20:14:50.079791 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 20:14:50.079791 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 20:14:50.079791 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 20:14:50.079791 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 20:14:50.079791 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 20:14:50.079791 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 20:14:50.079791 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception Sep 09 20:14:50.080633 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 20:14:50.080633 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 20:14:50.080633 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 20:14:50.080633 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 20:14:50.080633 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 20:14:50.080633 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 20:14:50.080633 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 20:14:50.080633 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application conn.query(q) Sep 09 20:14:50.080633 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 20:14:50.080633 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 20:14:50.080633 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 20:14:50.080633 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application result.read() Sep 09 20:14:50.080633 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 20:14:50.080633 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 20:14:50.080633 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 20:14:50.080633 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 20:14:50.082733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 20:14:50.082733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 20:14:50.082733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 20:14:50.082733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 20:14:50.082733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 20:14:50.082733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 09 20:14:50.082733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [parameters: {'id': '3e9f2d8c3c8e44469eb935681a990867', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 09 20:14:50.082733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 20:14:50.082733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:14:50.082733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:14:50.082733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:14:50.082733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:50.082733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:50.082733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:50.082733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:50.082733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:50.083808 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:50.083808 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:50.083808 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:50.083808 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:50.083808 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:50.083808 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:50.083808 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 09 20:14:50.083808 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 09 20:14:50.083808 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:50.083808 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:50.083808 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 09 20:14:50.083808 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 09 20:14:50.083808 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 20:14:50.083808 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 20:14:50.083808 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 09 20:14:50.083808 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:14:50.085170 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 233/478] 10.222.0.26 () {60 vars in 1131 bytes} [Fri Sep 9 20:14:49 2022] POST /identity/v3/roles => generated 144 bytes in 187 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:50.085290 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2421be-da92-4780-994e-fa8b63108640 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:50.091889 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:50.202268 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2421be-da92-4780-994e-fa8b63108640 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:50.238223 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d53f73-cabd-4c02-8f1a-6f96d5533b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:50.238908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d53f73-cabd-4c02-8f1a-6f96d5533b7d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:50.239199 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d53f73-cabd-4c02-8f1a-6f96d5533b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:50.239475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d53f73-cabd-4c02-8f1a-6f96d5533b7d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:50.251431 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d53f73-cabd-4c02-8f1a-6f96d5533b7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:50.257747 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-73d53f73-cabd-4c02-8f1a-6f96d5533b7d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:14:50.257747 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:50.257747 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:50.257747 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:50.257747 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:50.257747 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:50.257747 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:50.257747 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:50.257747 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:50.257747 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:50.257747 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:50.257747 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:50.257747 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:50.257747 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:50.257747 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:50.257747 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:50.258713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:50.258713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:50.258713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:50.258713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:50.258713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:50.258713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:50.258713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:50.258713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:50.258713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:50.258713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:50.258713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:50.258713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:50.258713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:50.258713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:50.258713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:50.258713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:50.259549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:50.259549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:50.259549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:50.259549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:50.259549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:50.259549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:50.259549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:50.259549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:50.259549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:50.259549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:50.259549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:14:50.259549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:50.261372 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 246/479] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Sep 9 20:14:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:50.266239 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c375cc88-565d-4810-b2d5-6cd3c4558433 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:50.383135 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c375cc88-565d-4810-b2d5-6cd3c4558433 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:50.419166 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c6bfb7-b147-4781-8903-11d10939196b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:50.419783 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6bfb7-b147-4781-8903-11d10939196b None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:50.420065 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6bfb7-b147-4781-8903-11d10939196b None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:50.420341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6bfb7-b147-4781-8903-11d10939196b None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:50.421018 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c6bfb7-b147-4781-8903-11d10939196b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:50.421437 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c6bfb7-b147-4781-8903-11d10939196b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:50.422821 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c6bfb7-b147-4781-8903-11d10939196b None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:50.434958 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 234/480] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:14:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:51.883492 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77ecfa-b88c-48eb-a0cb-4314c8915e37 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:51.884527 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77ecfa-b88c-48eb-a0cb-4314c8915e37 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:51.885018 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77ecfa-b88c-48eb-a0cb-4314c8915e37 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:51.887014 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 247/481] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:51.892858 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34006a97-6f0e-4955-9746-3bc7f2a0bf8c None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:51.893865 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34006a97-6f0e-4955-9746-3bc7f2a0bf8c None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:51.894340 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34006a97-6f0e-4955-9746-3bc7f2a0bf8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:51.918289 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-34006a97-6f0e-4955-9746-3bc7f2a0bf8c None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:51.919481 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-34006a97-6f0e-4955-9746-3bc7f2a0bf8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:51.973776 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:51.992962 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 235/482] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:52.018950 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3904a6f9-d6f7-4062-8ef5-3573cff87cbc None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:52.019237 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3904a6f9-d6f7-4062-8ef5-3573cff87cbc None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:52.019746 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3904a6f9-d6f7-4062-8ef5-3573cff87cbc None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:52.022157 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 248/483] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:52.027164 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6d345-d86e-4608-b3ea-3b566ff75925 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:52.027448 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6d345-d86e-4608-b3ea-3b566ff75925 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:52.027721 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6d345-d86e-4608-b3ea-3b566ff75925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:52.049842 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-81d6d345-d86e-4608-b3ea-3b566ff75925 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:52.051008 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-81d6d345-d86e-4608-b3ea-3b566ff75925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:52.146672 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 236/484] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:52.328755 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3904a6f9-d6f7-4062-8ef5-3573cff87cbc None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:52.412119 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:52.465980 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3904a6f9-d6f7-4062-8ef5-3573cff87cbc None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:52.506998 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d6d345-d86e-4608-b3ea-3b566ff75925 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:52.527560 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afce3506-564e-43f7-a091-d27b09ae12c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:52.528637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce3506-564e-43f7-a091-d27b09ae12c2 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:52.528916 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce3506-564e-43f7-a091-d27b09ae12c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:52.529471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce3506-564e-43f7-a091-d27b09ae12c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:52.552099 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afce3506-564e-43f7-a091-d27b09ae12c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:52.557471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-afce3506-564e-43f7-a091-d27b09ae12c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:14:52.557471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:52.557471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:52.557471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:52.557471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:52.557471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:52.557471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:52.557471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:52.557471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:52.557471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:52.557471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:52.557471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:52.557471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:52.557471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:52.557471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:52.557471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:52.558369 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:52.558369 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:52.558369 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:52.558369 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:52.558369 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:52.558369 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:52.558369 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:52.558369 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:52.558369 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:52.558369 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:52.558369 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:52.558369 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:52.558369 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:52.558369 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:52.558369 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:52.558369 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:52.559200 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:52.559200 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:52.559200 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:52.559200 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:52.559200 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:52.559200 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:52.559200 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:52.559200 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:52.559200 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:52.559200 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:52.559200 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:14:52.559200 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:52.560835 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 249/485] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Sep 9 20:14:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:52.565543 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afce3506-564e-43f7-a091-d27b09ae12c2 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:52.643815 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d6d345-d86e-4608-b3ea-3b566ff75925 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:52.665996 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:52.694868 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc2fa60-6fb9-4058-b3c5-7c054ab30755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:52.695830 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2fa60-6fb9-4058-b3c5-7c054ab30755 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:52.696300 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2fa60-6fb9-4058-b3c5-7c054ab30755 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:52.696553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afce3506-564e-43f7-a091-d27b09ae12c2 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:52.696994 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2fa60-6fb9-4058-b3c5-7c054ab30755 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:52.711469 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc2fa60-6fb9-4058-b3c5-7c054ab30755 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:52.717985 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-8cc2fa60-6fb9-4058-b3c5-7c054ab30755 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:14:52.717985 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:52.717985 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:52.717985 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:52.717985 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:52.717985 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:52.717985 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:52.717985 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:52.717985 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:52.717985 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:52.717985 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:52.717985 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:52.717985 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:52.717985 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:52.717985 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:52.717985 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:52.718890 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:52.718890 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:52.718890 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:52.718890 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:52.718890 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:52.718890 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:52.718890 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:52.718890 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:52.718890 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:52.718890 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:52.718890 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:52.718890 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:52.718890 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:52.718890 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:52.718890 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:52.718890 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:52.719690 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:52.719690 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:52.719690 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:52.719690 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:52.719690 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:52.719690 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:52.719690 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:52.719690 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:52.719690 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:52.719690 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:52.719690 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:14:52.719690 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:14:52.723150 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 237/486] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Sep 9 20:14:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:52.724356 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:52.728108 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c4743-f587-44be-8b4f-fde55dddc265 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:52.729033 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c4743-f587-44be-8b4f-fde55dddc265 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:52.729531 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c4743-f587-44be-8b4f-fde55dddc265 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:52.731332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 238/487] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:52.732246 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75c4743-f587-44be-8b4f-fde55dddc265 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:52.751502 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a43029-8672-497d-a0b5-47caab27642b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:52.752131 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a43029-8672-497d-a0b5-47caab27642b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:52.752407 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a43029-8672-497d-a0b5-47caab27642b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:52.752678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a43029-8672-497d-a0b5-47caab27642b None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:52.753416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a43029-8672-497d-a0b5-47caab27642b None admin] RBAC: Adding query filter params (name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:52.753717 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a43029-8672-497d-a0b5-47caab27642b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:52.755391 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a43029-8672-497d-a0b5-47caab27642b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:52.770151 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 250/488] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Sep 9 20:14:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:52.770151 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283124f-01c9-4c72-b287-671358cd3bcc None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:52.770151 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283124f-01c9-4c72-b287-671358cd3bcc None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:52.770151 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283124f-01c9-4c72-b287-671358cd3bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:52.787308 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e283124f-01c9-4c72-b287-671358cd3bcc None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:52.789044 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e283124f-01c9-4c72-b287-671358cd3bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:52.864728 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75c4743-f587-44be-8b4f-fde55dddc265 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:52.871766 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 251/489] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:52.873463 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283124f-01c9-4c72-b287-671358cd3bcc None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:52.901680 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72085d3c-8a3b-4fd9-bf0c-e9a669f49f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:52.902186 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72085d3c-8a3b-4fd9-bf0c-e9a669f49f41 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:52.902462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72085d3c-8a3b-4fd9-bf0c-e9a669f49f41 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:52.902730 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72085d3c-8a3b-4fd9-bf0c-e9a669f49f41 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:52.903802 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72085d3c-8a3b-4fd9-bf0c-e9a669f49f41 None admin] RBAC: Adding query filter params (name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:52.904125 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72085d3c-8a3b-4fd9-bf0c-e9a669f49f41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:52.905888 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72085d3c-8a3b-4fd9-bf0c-e9a669f49f41 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:52.921646 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 239/490] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Sep 9 20:14:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:52.923755 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26a315-584c-47d7-af79-b294f229f0ba None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:52.924044 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26a315-584c-47d7-af79-b294f229f0ba None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:52.924318 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26a315-584c-47d7-af79-b294f229f0ba None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:52.925889 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 240/491] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:52.926585 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be26a315-584c-47d7-af79-b294f229f0ba None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:52.978022 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:53.012558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283124f-01c9-4c72-b287-671358cd3bcc None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:53.032121 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:53.056561 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be26a315-584c-47d7-af79-b294f229f0ba None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:53.061879 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a79ad4-fbfe-41a6-b51b-b99da36560af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:53.062539 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a79ad4-fbfe-41a6-b51b-b99da36560af None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:53.063093 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a79ad4-fbfe-41a6-b51b-b99da36560af None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:53.063366 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a79ad4-fbfe-41a6-b51b-b99da36560af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:53.076758 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a79ad4-fbfe-41a6-b51b-b99da36560af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:53.080075 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a79ad4-fbfe-41a6-b51b-b99da36560af None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:53.100603 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-68a79ad4-fbfe-41a6-b51b-b99da36560af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:14:53.100603 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:53.100603 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:53.100603 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:53.100603 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:53.100603 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:53.100603 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:53.100603 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:53.100603 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:53.100603 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:53.100603 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:53.100603 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:53.100603 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:14:53.100603 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:14:53.100603 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:14:53.100603 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:14:53.101610 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:53.101610 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:53.101610 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:53.101610 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:53.101610 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:53.101610 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:53.101610 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:53.101610 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:53.101610 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:53.101610 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:53.101610 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:53.101610 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:53.101610 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:53.101610 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:53.101610 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:53.101610 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:53.101610 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:14:53.102529 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:14:53.102529 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:14:53.102529 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:53.102529 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 252/492] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:14:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:53.103214 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f559b-e744-40ef-bbe4-078dd5e2e1bd None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:53.103308 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f559b-e744-40ef-bbe4-078dd5e2e1bd None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:53.103539 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f559b-e744-40ef-bbe4-078dd5e2e1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:53.105761 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c468e3c-752b-4d03-94c2-9cc592294ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:53.106632 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c468e3c-752b-4d03-94c2-9cc592294ddd None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:53.107108 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c468e3c-752b-4d03-94c2-9cc592294ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:53.107565 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c468e3c-752b-4d03-94c2-9cc592294ddd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:53.120348 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c468e3c-752b-4d03-94c2-9cc592294ddd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:53.122097 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c468e3c-752b-4d03-94c2-9cc592294ddd None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:53.123304 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-d15f559b-e744-40ef-bbe4-078dd5e2e1bd None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:53.124476 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d15f559b-e744-40ef-bbe4-078dd5e2e1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:53.138705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-7c468e3c-752b-4d03-94c2-9cc592294ddd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:14:53.138705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:53.138705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:53.138705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:53.138705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:53.138705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:53.138705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:53.138705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:53.138705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:53.138705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:53.138705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:53.138705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:53.138705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:14:53.138705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:14:53.138705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:14:53.138705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:14:53.139677 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:53.139677 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:53.139677 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:53.139677 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:53.139677 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:53.139677 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:53.139677 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:53.139677 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:53.139677 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:53.139677 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:53.139677 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:53.139677 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:53.139677 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:53.139677 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:53.139677 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:53.139677 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:53.139677 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:14:53.140616 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:14:53.140616 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:14:53.140616 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:14:53.142887 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 241/493] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:14:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:53.143863 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c468e3c-752b-4d03-94c2-9cc592294ddd None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:53.204036 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 253/494] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:53.204787 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f559b-e744-40ef-bbe4-078dd5e2e1bd None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:53.252330 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:53.264509 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c468e3c-752b-4d03-94c2-9cc592294ddd None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:53.299483 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac731891-0067-4bce-842e-c0ca54a21d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:53.300094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac731891-0067-4bce-842e-c0ca54a21d04 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:53.300370 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac731891-0067-4bce-842e-c0ca54a21d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:53.300630 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac731891-0067-4bce-842e-c0ca54a21d04 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:53.301284 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac731891-0067-4bce-842e-c0ca54a21d04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:53.301591 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac731891-0067-4bce-842e-c0ca54a21d04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:53.302941 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac731891-0067-4bce-842e-c0ca54a21d04 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:53.308195 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:53.315062 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f559b-e744-40ef-bbe4-078dd5e2e1bd None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:53.315209 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 242/495] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:53.319961 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac731891-0067-4bce-842e-c0ca54a21d04 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:53.356337 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c877275-294a-476b-aa1c-5caccc646893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:53.357028 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c877275-294a-476b-aa1c-5caccc646893 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:53.357418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c877275-294a-476b-aa1c-5caccc646893 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:53.357756 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c877275-294a-476b-aa1c-5caccc646893 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:53.358485 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c877275-294a-476b-aa1c-5caccc646893 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:53.358843 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c877275-294a-476b-aa1c-5caccc646893 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:53.360269 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c877275-294a-476b-aa1c-5caccc646893 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:53.376012 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 254/496] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:53.376012 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4d539-9c0d-41c8-a98c-b00469c31069 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:53.376012 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4d539-9c0d-41c8-a98c-b00469c31069 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:53.376012 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4d539-9c0d-41c8-a98c-b00469c31069 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:53.376012 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 255/497] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:53.376012 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b4d539-9c0d-41c8-a98c-b00469c31069 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:53.426983 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac731891-0067-4bce-842e-c0ca54a21d04 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:53.464629 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c0810-38da-47b3-9184-8a5ca8398b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:53.465278 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c0810-38da-47b3-9184-8a5ca8398b7b None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:53.465558 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c0810-38da-47b3-9184-8a5ca8398b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:53.465886 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c0810-38da-47b3-9184-8a5ca8398b7b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:53.476645 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182c0810-38da-47b3-9184-8a5ca8398b7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:53.481484 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-182c0810-38da-47b3-9184-8a5ca8398b7b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:14:53.481484 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:53.481484 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:53.481484 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:53.481484 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:53.481484 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:53.481484 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:53.481484 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:53.481484 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:53.481484 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:53.481484 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:53.481484 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:53.481484 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:53.481484 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:53.481484 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:53.481484 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:53.482546 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:53.482546 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:53.482546 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:53.482546 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:53.482546 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:53.482546 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:53.482546 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:53.482546 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:53.482546 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:53.482546 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:53.482546 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:53.482546 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:53.482546 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:53.482546 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:53.482546 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:53.482546 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:53.483713 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:53.483713 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:53.483713 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:53.483713 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:53.483713 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:53.483713 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:53.483713 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:53.483713 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:53.483713 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:53.483713 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:53.483713 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:14:53.483713 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:14:53.483713 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 243/498] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Sep 9 20:14:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:53.483713 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c0810-38da-47b3-9184-8a5ca8398b7b None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:53.490957 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b4d539-9c0d-41c8-a98c-b00469c31069 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:53.496314 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:53.525255 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a447535-37b7-4ce8-bade-5acb9af7e31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:53.525901 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a447535-37b7-4ce8-bade-5acb9af7e31c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:53.526176 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a447535-37b7-4ce8-bade-5acb9af7e31c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:53.526444 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a447535-37b7-4ce8-bade-5acb9af7e31c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:53.533337 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a447535-37b7-4ce8-bade-5acb9af7e31c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:53.534769 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a447535-37b7-4ce8-bade-5acb9af7e31c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:53.547585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-4a447535-37b7-4ce8-bade-5acb9af7e31c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:14:53.547585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:53.547585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:53.547585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:53.547585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:53.547585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:53.547585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:53.547585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:53.547585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:53.547585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:53.547585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:53.547585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:53.547585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:14:53.547585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:14:53.547585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:14:53.547585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:14:53.548744 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:53.548744 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:53.548744 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:14:53.548744 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:14:53.548744 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:14:53.548744 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:14:53.548744 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:53.548744 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:53.548744 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:53.548744 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:53.548744 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:53.548744 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:53.548744 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:53.548744 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:53.548744 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:53.548744 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:53.548744 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:53.548744 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:53.549913 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:53.549913 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:53.549913 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:14:53.549913 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:14:53.549913 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:14:53.549913 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:14:53.549913 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:14:53.549913 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:14:53.549913 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:14:53.549913 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:53.553425 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 256/499] 10.222.0.26 () {56 vars in 1093 bytes} [Fri Sep 9 20:14:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:53.554574 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64faf37-86d1-432d-9f16-feb9b55df5ba None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:53.554856 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64faf37-86d1-432d-9f16-feb9b55df5ba None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:53.555273 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64faf37-86d1-432d-9f16-feb9b55df5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:53.574324 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-b64faf37-86d1-432d-9f16-feb9b55df5ba None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:53.575508 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b64faf37-86d1-432d-9f16-feb9b55df5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:53.587216 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:53.607981 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c0810-38da-47b3-9184-8a5ca8398b7b None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:53.641384 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 257/500] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:53.641525 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f979f9-c544-4a61-8775-40bb0d868081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:53.642052 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f979f9-c544-4a61-8775-40bb0d868081 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:53.642216 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64faf37-86d1-432d-9f16-feb9b55df5ba None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:53.642310 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f979f9-c544-4a61-8775-40bb0d868081 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:53.642583 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f979f9-c544-4a61-8775-40bb0d868081 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:53.659742 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f979f9-c544-4a61-8775-40bb0d868081 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:53.661209 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f979f9-c544-4a61-8775-40bb0d868081 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:53.677023 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-15f979f9-c544-4a61-8775-40bb0d868081 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 20:14:53.677023 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:53.677023 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:53.677023 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:53.677023 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:53.677023 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:53.677023 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:53.677023 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:53.677023 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:53.677023 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:53.677023 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:53.677023 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:53.677023 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:14:53.677023 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:14:53.677023 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:14:53.677023 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:14:53.677964 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:53.677964 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:53.677964 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:14:53.677964 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:14:53.677964 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:14:53.677964 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:14:53.677964 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:53.677964 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:53.677964 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:53.677964 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:53.677964 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:53.677964 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:53.677964 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:53.677964 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:53.677964 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:53.677964 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:53.677964 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:53.677964 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:53.679428 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:53.679428 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:53.679428 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:14:53.679428 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:14:53.679428 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:14:53.679428 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:14:53.679428 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:14:53.679428 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:14:53.679428 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 20:14:53.679428 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:14:53.679428 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 244/501] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:14:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:53.679428 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f979f9-c544-4a61-8775-40bb0d868081 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:53.750039 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64faf37-86d1-432d-9f16-feb9b55df5ba None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:53.755960 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:53.789553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f3991-d0ed-4c85-b2d3-c092400e59a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:53.790114 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f979f9-c544-4a61-8775-40bb0d868081 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:53.790218 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f3991-d0ed-4c85-b2d3-c092400e59a0 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:53.790497 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f3991-d0ed-4c85-b2d3-c092400e59a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:53.790768 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f3991-d0ed-4c85-b2d3-c092400e59a0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:53.803191 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f3991-d0ed-4c85-b2d3-c092400e59a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:53.809206 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-3f3f3991-d0ed-4c85-b2d3-c092400e59a0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:14:53.809206 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:53.809206 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:53.809206 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:53.809206 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:53.809206 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:53.809206 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:53.809206 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:53.809206 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:53.809206 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:53.809206 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:53.809206 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:53.809206 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:53.809206 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:53.809206 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:53.809206 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:53.810170 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:53.810170 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:53.810170 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:53.810170 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:53.810170 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:53.810170 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:53.810170 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:53.810170 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:53.810170 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:53.810170 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:53.810170 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:53.810170 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:53.810170 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:53.810170 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:53.810170 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:53.810170 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:53.811020 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:53.811020 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:53.811020 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:53.811020 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:53.811020 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:53.811020 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:53.811020 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:53.811020 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:53.811020 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:53.811020 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:53.811020 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:14:53.811020 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:53.811020 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 258/502] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Sep 9 20:14:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:53.814103 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f3991-d0ed-4c85-b2d3-c092400e59a0 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:53.823883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447987ac-7385-4038-9027-8650f015dd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:53.824738 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:53.825321 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447987ac-7385-4038-9027-8650f015dd6f None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:53.825491 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447987ac-7385-4038-9027-8650f015dd6f None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:53.825767 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447987ac-7385-4038-9027-8650f015dd6f None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:53.826553 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447987ac-7385-4038-9027-8650f015dd6f None admin] RBAC: Adding query filter params (name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:53.827054 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447987ac-7385-4038-9027-8650f015dd6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:53.828717 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447987ac-7385-4038-9027-8650f015dd6f None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:53.842299 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 245/503] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Sep 9 20:14:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:53.842299 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447987ac-7385-4038-9027-8650f015dd6f None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:53.924434 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f3991-d0ed-4c85-b2d3-c092400e59a0 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:53.945552 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447987ac-7385-4038-9027-8650f015dd6f None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:53.957580 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140648df-3794-4508-b1b3-a3bfadd7afef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:53.958179 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140648df-3794-4508-b1b3-a3bfadd7afef None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:53.958454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140648df-3794-4508-b1b3-a3bfadd7afef None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:53.958723 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140648df-3794-4508-b1b3-a3bfadd7afef None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:53.959439 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140648df-3794-4508-b1b3-a3bfadd7afef None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:53.959740 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140648df-3794-4508-b1b3-a3bfadd7afef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:53.961180 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140648df-3794-4508-b1b3-a3bfadd7afef None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:53.973085 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:53.978808 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c8074-0ef7-415f-9f96-48cf121bc034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:53.979416 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c8074-0ef7-415f-9f96-48cf121bc034 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:53.979700 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c8074-0ef7-415f-9f96-48cf121bc034 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:53.979975 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c8074-0ef7-415f-9f96-48cf121bc034 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:53.980699 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c8074-0ef7-415f-9f96-48cf121bc034 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:53.981005 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c8074-0ef7-415f-9f96-48cf121bc034 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:53.982450 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c8074-0ef7-415f-9f96-48cf121bc034 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:53.997383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 259/504] 10.222.0.26 () {56 vars in 1133 bytes} [Fri Sep 9 20:14:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:53.999170 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140648df-3794-4508-b1b3-a3bfadd7afef None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:54.024508 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 246/505] 10.222.0.26 () {56 vars in 1137 bytes} [Fri Sep 9 20:14:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:54.024508 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c8074-0ef7-415f-9f96-48cf121bc034 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:54.061398 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:54.097709 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140648df-3794-4508-b1b3-a3bfadd7afef None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:54.117104 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c8074-0ef7-415f-9f96-48cf121bc034 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:54.131326 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70781f84-342e-4e3d-919a-6a132aebd94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:54.132445 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70781f84-342e-4e3d-919a-6a132aebd94b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:54.132725 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70781f84-342e-4e3d-919a-6a132aebd94b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:54.133002 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70781f84-342e-4e3d-919a-6a132aebd94b None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:54.133694 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70781f84-342e-4e3d-919a-6a132aebd94b None admin] RBAC: Adding query filter params (name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:54.133999 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70781f84-342e-4e3d-919a-6a132aebd94b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:54.135363 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70781f84-342e-4e3d-919a-6a132aebd94b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:54.143014 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 260/506] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Sep 9 20:14:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:54.143786 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70781f84-342e-4e3d-919a-6a132aebd94b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:54.155041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868077a6-14b5-456c-a28b-14c728fdec06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:54.155666 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868077a6-14b5-456c-a28b-14c728fdec06 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:54.155940 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868077a6-14b5-456c-a28b-14c728fdec06 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:54.156210 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868077a6-14b5-456c-a28b-14c728fdec06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:54.167032 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868077a6-14b5-456c-a28b-14c728fdec06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:54.168477 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868077a6-14b5-456c-a28b-14c728fdec06 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:54.182732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-868077a6-14b5-456c-a28b-14c728fdec06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:14:54.182732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:54.182732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:54.182732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:54.182732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:54.182732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:54.182732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:54.182732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:54.182732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:54.182732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:54.182732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:54.182732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:54.182732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:14:54.182732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:14:54.182732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:14:54.182732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:14:54.183701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:54.183701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:54.183701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:54.183701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:54.183701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:54.183701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:54.183701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:54.183701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:54.183701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:54.183701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:54.183701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:54.183701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:54.183701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:54.183701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:54.183701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:54.183701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:54.183701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:14:54.184570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:14:54.184570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:14:54.184570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:14:54.184570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 247/507] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:14:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:54.184570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868077a6-14b5-456c-a28b-14c728fdec06 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:54.231654 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:54.242466 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70781f84-342e-4e3d-919a-6a132aebd94b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:54.267325 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:54.277379 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35163265-83e8-425a-8a99-880a16819c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:54.277948 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35163265-83e8-425a-8a99-880a16819c8f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:54.278231 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35163265-83e8-425a-8a99-880a16819c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:54.278670 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35163265-83e8-425a-8a99-880a16819c8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:54.287217 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868077a6-14b5-456c-a28b-14c728fdec06 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:54.289833 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35163265-83e8-425a-8a99-880a16819c8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:54.291346 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35163265-83e8-425a-8a99-880a16819c8f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:54.305434 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-35163265-83e8-425a-8a99-880a16819c8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:14:54.305434 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:54.305434 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:54.305434 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:54.305434 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:54.305434 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:54.305434 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:54.305434 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:54.305434 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:54.305434 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:54.305434 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:54.305434 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:54.305434 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:14:54.305434 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:14:54.305434 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:14:54.305434 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:14:54.306553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:54.306553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:54.306553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:54.306553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:54.306553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:54.306553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:54.306553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:54.306553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:54.306553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:54.306553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:54.306553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:54.306553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:54.306553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:54.306553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:54.306553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:54.306553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:54.306553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:14:54.307419 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:14:54.307419 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:14:54.307419 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:54.307419 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 261/508] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:14:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:54.307419 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35163265-83e8-425a-8a99-880a16819c8f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:54.323014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980a101-c096-4e3c-8527-2586f2a36521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:54.323618 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980a101-c096-4e3c-8527-2586f2a36521 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:54.323903 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980a101-c096-4e3c-8527-2586f2a36521 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:54.324175 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980a101-c096-4e3c-8527-2586f2a36521 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:54.391321 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e980a101-c096-4e3c-8527-2586f2a36521 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:54.392782 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e980a101-c096-4e3c-8527-2586f2a36521 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:54.407732 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35163265-83e8-425a-8a99-880a16819c8f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:54.433299 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:54.441364 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d6dce-29cf-4650-9527-8f05cdfaa4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:54.441967 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d6dce-29cf-4650-9527-8f05cdfaa4df None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:54.442249 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d6dce-29cf-4650-9527-8f05cdfaa4df None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:54.442524 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d6dce-29cf-4650-9527-8f05cdfaa4df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:54.487829 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:54.489125 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 248/509] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:14:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:14:54.490001 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980a101-c096-4e3c-8527-2586f2a36521 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:54.511166 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d6dce-29cf-4650-9527-8f05cdfaa4df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:54.512621 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d6dce-29cf-4650-9527-8f05cdfaa4df None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:54.586604 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980a101-c096-4e3c-8527-2586f2a36521 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:54.609899 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 262/510] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:14:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:14:54.611011 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d6dce-29cf-4650-9527-8f05cdfaa4df None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:54.624464 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be84ee5-2ece-40fb-b11d-080520d3b7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:54.625090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be84ee5-2ece-40fb-b11d-080520d3b7c1 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:54.625402 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be84ee5-2ece-40fb-b11d-080520d3b7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:54.625681 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be84ee5-2ece-40fb-b11d-080520d3b7c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:54.636966 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be84ee5-2ece-40fb-b11d-080520d3b7c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:54.638447 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be84ee5-2ece-40fb-b11d-080520d3b7c1 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:54.640178 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:54.654681 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-3be84ee5-2ece-40fb-b11d-080520d3b7c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:14:54.654681 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:54.654681 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:54.654681 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:54.654681 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:54.654681 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:54.654681 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:54.654681 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:54.654681 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:54.654681 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:54.654681 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:54.654681 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:54.654681 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:14:54.654681 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:14:54.654681 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:14:54.654681 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:14:54.655635 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:54.655635 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:54.655635 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:54.655635 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:54.655635 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:54.655635 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:54.655635 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:54.655635 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:54.655635 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:54.655635 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:54.655635 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:54.655635 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:54.655635 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:54.655635 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:54.655635 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:54.655635 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:54.655635 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:14:54.656501 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:14:54.656501 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:14:54.656501 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:14:54.656501 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 249/511] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:14:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:54.657130 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be84ee5-2ece-40fb-b11d-080520d3b7c1 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:54.695808 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:54.718428 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d6dce-29cf-4650-9527-8f05cdfaa4df None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:54.751950 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f5e40-e8a4-4ba4-834a-e2d6ff3ba0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:54.752556 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f5e40-e8a4-4ba4-834a-e2d6ff3ba0d1 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:54.752831 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f5e40-e8a4-4ba4-834a-e2d6ff3ba0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:54.753100 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f5e40-e8a4-4ba4-834a-e2d6ff3ba0d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:54.753774 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10f5e40-e8a4-4ba4-834a-e2d6ff3ba0d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:54.754331 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10f5e40-e8a4-4ba4-834a-e2d6ff3ba0d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:54.755687 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10f5e40-e8a4-4ba4-834a-e2d6ff3ba0d1 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:54.758234 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be84ee5-2ece-40fb-b11d-080520d3b7c1 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:54.766669 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 263/512] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:14:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:54.767455 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f5e40-e8a4-4ba4-834a-e2d6ff3ba0d1 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:54.796029 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00b4280-b0a3-4d51-9839-d3a3d16f9117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:54.796652 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b4280-b0a3-4d51-9839-d3a3d16f9117 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:54.796932 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b4280-b0a3-4d51-9839-d3a3d16f9117 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:54.797204 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b4280-b0a3-4d51-9839-d3a3d16f9117 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:54.797878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00b4280-b0a3-4d51-9839-d3a3d16f9117 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:54.798179 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00b4280-b0a3-4d51-9839-d3a3d16f9117 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:54.799549 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00b4280-b0a3-4d51-9839-d3a3d16f9117 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:54.812132 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 250/513] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:14:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:54.812837 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00b4280-b0a3-4d51-9839-d3a3d16f9117 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:54.867016 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:54.874157 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f5e40-e8a4-4ba4-834a-e2d6ff3ba0d1 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:54.907627 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4051264-f893-4951-a451-e1c7eed9e36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:54.908248 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4051264-f893-4951-a451-e1c7eed9e36c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:54.908526 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4051264-f893-4951-a451-e1c7eed9e36c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:54.908796 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4051264-f893-4951-a451-e1c7eed9e36c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:54.910474 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00b4280-b0a3-4d51-9839-d3a3d16f9117 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:54.916219 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:54.921223 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4051264-f893-4951-a451-e1c7eed9e36c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:54.922713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4051264-f893-4951-a451-e1c7eed9e36c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:54.938226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-b4051264-f893-4951-a451-e1c7eed9e36c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:14:54.938226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:54.938226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:54.938226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:54.938226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:54.938226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:54.938226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:54.938226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:54.938226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:54.938226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:54.938226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:54.938226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:54.938226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:14:54.938226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:14:54.938226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:14:54.938226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:14:54.940110 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:54.940110 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:54.940110 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:54.940110 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:54.940110 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:54.940110 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:54.940110 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:54.940110 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:54.940110 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:54.940110 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:54.940110 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:54.940110 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:54.940110 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:54.940110 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:54.940110 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:54.940110 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:54.940110 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:14:54.941227 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:14:54.941227 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:14:54.941227 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:54.941227 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 264/514] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:14:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:54.941227 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4051264-f893-4951-a451-e1c7eed9e36c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:54.954968 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95322db-321c-416b-aaa6-4772946eed5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:54.955598 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95322db-321c-416b-aaa6-4772946eed5e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:54.955906 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95322db-321c-416b-aaa6-4772946eed5e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:54.956140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95322db-321c-416b-aaa6-4772946eed5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:54.966143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95322db-321c-416b-aaa6-4772946eed5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:54.967585 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95322db-321c-416b-aaa6-4772946eed5e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:54.982295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-b95322db-321c-416b-aaa6-4772946eed5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:14:54.982295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:54.982295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:54.982295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:54.982295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:54.982295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:54.982295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:54.982295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:54.982295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:54.982295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:54.982295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:54.982295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:54.982295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:14:54.982295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:14:54.982295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:14:54.982295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:14:54.983215 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:54.983215 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:54.983215 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:54.983215 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:54.983215 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:54.983215 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:54.983215 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:54.983215 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:54.983215 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:54.983215 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:54.983215 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:54.983215 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:54.983215 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:54.983215 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:54.983215 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:54.983215 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:54.983215 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:14:54.984123 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:14:54.984123 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:14:54.984123 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:14:54.984123 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 251/515] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:14:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:54.984123 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95322db-321c-416b-aaa6-4772946eed5e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:55.044756 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4051264-f893-4951-a451-e1c7eed9e36c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:55.077296 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b6cbd9-eb5c-44d2-b62d-8b5d96dc5797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:55.077884 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b6cbd9-eb5c-44d2-b62d-8b5d96dc5797 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:55.078168 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b6cbd9-eb5c-44d2-b62d-8b5d96dc5797 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:55.078449 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b6cbd9-eb5c-44d2-b62d-8b5d96dc5797 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:55.079094 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b6cbd9-eb5c-44d2-b62d-8b5d96dc5797 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:55.079401 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b6cbd9-eb5c-44d2-b62d-8b5d96dc5797 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:55.080772 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b6cbd9-eb5c-44d2-b62d-8b5d96dc5797 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:55.085716 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95322db-321c-416b-aaa6-4772946eed5e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:55.095103 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 265/516] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:14:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:55.095907 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b6cbd9-eb5c-44d2-b62d-8b5d96dc5797 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:55.098741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:55.126295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306a7c4-831e-4ccd-94f4-dc8bdf552f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:55.126860 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306a7c4-831e-4ccd-94f4-dc8bdf552f4e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:55.127140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306a7c4-831e-4ccd-94f4-dc8bdf552f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:55.127420 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306a7c4-831e-4ccd-94f4-dc8bdf552f4e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:55.135545 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a306a7c4-831e-4ccd-94f4-dc8bdf552f4e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:55.136990 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a306a7c4-831e-4ccd-94f4-dc8bdf552f4e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:55.148471 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-a306a7c4-831e-4ccd-94f4-dc8bdf552f4e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 20:14:55.148471 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:55.148471 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:55.148471 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:55.148471 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:55.148471 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:55.148471 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:55.148471 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:55.148471 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:55.148471 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:55.148471 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:55.148471 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:55.148471 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:14:55.148471 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:14:55.148471 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:14:55.148471 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:14:55.149407 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:55.149407 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:55.149407 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:14:55.149407 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:14:55.149407 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:14:55.149407 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:14:55.149407 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:55.149407 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:55.149407 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:55.149407 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:55.149407 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:55.149407 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:55.149407 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:55.149407 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:55.149407 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:55.149407 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:55.149407 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:55.149407 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:55.150335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:55.150335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:55.150335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:14:55.150335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:14:55.150335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:14:55.150335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:14:55.150335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:14:55.150335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:14:55.150335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 20:14:55.150335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:14:55.150854 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:55.150924 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 252/517] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Sep 9 20:14:54 2022] GET /identity/v3/users/neutron => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:55.150924 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306a7c4-831e-4ccd-94f4-dc8bdf552f4e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:55.204289 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b6cbd9-eb5c-44d2-b62d-8b5d96dc5797 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:55.236669 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374324e-8050-4a68-9b81-db39ec6f57e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:55.237292 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374324e-8050-4a68-9b81-db39ec6f57e4 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:55.237579 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374324e-8050-4a68-9b81-db39ec6f57e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:55.237858 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374324e-8050-4a68-9b81-db39ec6f57e4 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:55.238498 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c374324e-8050-4a68-9b81-db39ec6f57e4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:55.239928 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c374324e-8050-4a68-9b81-db39ec6f57e4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:55.247083 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306a7c4-831e-4ccd-94f4-dc8bdf552f4e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:55.261390 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 266/518] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Sep 9 20:14:55 2022] POST /identity/v3/users => generated 247 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:14:55.262211 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374324e-8050-4a68-9b81-db39ec6f57e4 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:55.303860 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3495e5bd-1a13-45ee-a50a-0b653aafba16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:55.304539 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3495e5bd-1a13-45ee-a50a-0b653aafba16 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:55.304828 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3495e5bd-1a13-45ee-a50a-0b653aafba16 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:55.305107 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3495e5bd-1a13-45ee-a50a-0b653aafba16 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:55.305796 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3495e5bd-1a13-45ee-a50a-0b653aafba16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:55.306433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3495e5bd-1a13-45ee-a50a-0b653aafba16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:55.307834 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3495e5bd-1a13-45ee-a50a-0b653aafba16 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:55.326086 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 253/519] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:14:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:55.326086 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3495e5bd-1a13-45ee-a50a-0b653aafba16 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:55.358693 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:55.382973 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374324e-8050-4a68-9b81-db39ec6f57e4 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:55.392121 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:55.426145 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d2ad8d-d579-4805-bea6-d542f818f80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:55.426782 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d2ad8d-d579-4805-bea6-d542f818f80c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:55.427096 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d2ad8d-d579-4805-bea6-d542f818f80c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:55.427453 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d2ad8d-d579-4805-bea6-d542f818f80c None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:55.428112 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d2ad8d-d579-4805-bea6-d542f818f80c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:55.428416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d2ad8d-d579-4805-bea6-d542f818f80c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:55.429875 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d2ad8d-d579-4805-bea6-d542f818f80c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:55.441373 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3495e5bd-1a13-45ee-a50a-0b653aafba16 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:55.442635 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 267/520] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:14:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:55.443021 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d2ad8d-d579-4805-bea6-d542f818f80c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:55.485599 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725f481-21db-411f-ba9f-808dc053f0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:55.486365 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725f481-21db-411f-ba9f-808dc053f0b2 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:55.486641 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725f481-21db-411f-ba9f-808dc053f0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:55.486909 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725f481-21db-411f-ba9f-808dc053f0b2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:55.495307 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f725f481-21db-411f-ba9f-808dc053f0b2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:55.496751 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f725f481-21db-411f-ba9f-808dc053f0b2 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:55.508999 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-f725f481-21db-411f-ba9f-808dc053f0b2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 20:14:55.508999 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:55.508999 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:55.508999 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:55.508999 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:55.508999 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:55.508999 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:55.508999 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:55.508999 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:55.508999 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:55.508999 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:55.508999 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:55.508999 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:14:55.508999 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:14:55.508999 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:14:55.508999 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:14:55.510224 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:55.510224 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:55.510224 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:14:55.510224 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:14:55.510224 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:14:55.510224 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:14:55.510224 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:55.510224 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:55.510224 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:55.510224 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:55.510224 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:55.510224 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:55.510224 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:55.510224 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:55.510224 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:55.510224 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:55.510224 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:55.510224 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:55.511714 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:55.511714 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:55.511714 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:14:55.511714 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:14:55.511714 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:14:55.511714 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:14:55.511714 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:14:55.511714 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:14:55.511714 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 20:14:55.511714 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:14:55.511714 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 254/521] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:14:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:55.511714 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725f481-21db-411f-ba9f-808dc053f0b2 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:55.560950 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d2ad8d-d579-4805-bea6-d542f818f80c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:55.595427 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed26ba77-3ca5-4667-8a07-2c31c1bf7818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:55.596035 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26ba77-3ca5-4667-8a07-2c31c1bf7818 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:55.596311 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26ba77-3ca5-4667-8a07-2c31c1bf7818 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:55.596580 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26ba77-3ca5-4667-8a07-2c31c1bf7818 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:55.597533 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed26ba77-3ca5-4667-8a07-2c31c1bf7818 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:55.597641 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed26ba77-3ca5-4667-8a07-2c31c1bf7818 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:55.599035 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed26ba77-3ca5-4667-8a07-2c31c1bf7818 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:55.610532 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:55.622743 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 268/522] 10.222.0.26 () {56 vars in 1139 bytes} [Fri Sep 9 20:14:55 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:55.622743 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed26ba77-3ca5-4667-8a07-2c31c1bf7818 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:55.639484 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725f481-21db-411f-ba9f-808dc053f0b2 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:55.647349 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:55.684599 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63387c5-53ce-4b68-a74c-df54b09681c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:55.685349 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63387c5-53ce-4b68-a74c-df54b09681c6 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:55.685645 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63387c5-53ce-4b68-a74c-df54b09681c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:55.686013 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63387c5-53ce-4b68-a74c-df54b09681c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:55.692707 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63387c5-53ce-4b68-a74c-df54b09681c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:55.694206 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63387c5-53ce-4b68-a74c-df54b09681c6 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:55.706176 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-d63387c5-53ce-4b68-a74c-df54b09681c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:14:55.706176 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:55.706176 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:55.706176 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:55.706176 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:55.706176 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:55.706176 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:55.706176 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:55.706176 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:55.706176 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:55.706176 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:55.706176 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:55.706176 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:14:55.706176 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:14:55.706176 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:14:55.707102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:14:55.707102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:55.707102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:55.707102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:55.707102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:55.707102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:55.707102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:55.707102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:55.707102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:55.707102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:55.707102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:55.707102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:55.707102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:55.707102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:55.707102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:55.707102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:55.707102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:55.707979 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:14:55.707979 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:14:55.707979 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:14:55.707979 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:14:55.707979 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:14:55.707979 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:14:55.707979 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:14:55.707979 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:14:55.707979 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 255/523] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:14:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:55.707979 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63387c5-53ce-4b68-a74c-df54b09681c6 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:55.735053 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed26ba77-3ca5-4667-8a07-2c31c1bf7818 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:55.771020 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4540b8e-b140-4c65-ae2c-ce6669874eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:55.771649 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4540b8e-b140-4c65-ae2c-ce6669874eeb None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:55.772077 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4540b8e-b140-4c65-ae2c-ce6669874eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:55.772397 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4540b8e-b140-4c65-ae2c-ce6669874eeb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:55.782729 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4540b8e-b140-4c65-ae2c-ce6669874eeb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:55.784941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4540b8e-b140-4c65-ae2c-ce6669874eeb None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:55.799956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-b4540b8e-b140-4c65-ae2c-ce6669874eeb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:14:55.799956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:55.799956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:55.799956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:55.799956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:55.799956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:55.799956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:55.799956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:55.799956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:55.799956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:55.799956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:55.799956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:55.799956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:14:55.799956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:14:55.799956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:14:55.801120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:14:55.801120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:55.801120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:55.801120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:55.801120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:55.801120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:55.801120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:55.801120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:55.801120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:55.801120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:55.801120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:55.801120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:55.801120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:55.801120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:55.801120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:55.801120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:55.801120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:55.802033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:14:55.802033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:14:55.802033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:14:55.802033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:14:55.802033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:14:55.802033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:14:55.802033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:14:55.802033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:55.802033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 269/524] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:14:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:55.802503 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4540b8e-b140-4c65-ae2c-ce6669874eeb None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:55.819015 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63387c5-53ce-4b68-a74c-df54b09681c6 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:55.855247 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea9ff2-fa7a-4aa0-bdb8-853084395f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:55.855978 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea9ff2-fa7a-4aa0-bdb8-853084395f56 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:55.856319 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea9ff2-fa7a-4aa0-bdb8-853084395f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:55.856661 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea9ff2-fa7a-4aa0-bdb8-853084395f56 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:55.857516 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ea9ff2-fa7a-4aa0-bdb8-853084395f56 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:55.857888 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ea9ff2-fa7a-4aa0-bdb8-853084395f56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:55.859365 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ea9ff2-fa7a-4aa0-bdb8-853084395f56 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:55.883395 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 256/525] 10.222.0.26 () {56 vars in 1137 bytes} [Fri Sep 9 20:14:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:55.883395 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea9ff2-fa7a-4aa0-bdb8-853084395f56 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:55.888719 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:55.901527 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:55.919746 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4540b8e-b140-4c65-ae2c-ce6669874eeb None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:55.961447 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82764a5d-aa9c-4323-b904-a9929d769288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:55.962299 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82764a5d-aa9c-4323-b904-a9929d769288 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:55.962578 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82764a5d-aa9c-4323-b904-a9929d769288 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:55.962844 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82764a5d-aa9c-4323-b904-a9929d769288 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:56.000364 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea9ff2-fa7a-4aa0-bdb8-853084395f56 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:56.034940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82764a5d-aa9c-4323-b904-a9929d769288 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:56.036000 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1485215-c232-4fde-803b-457a40123f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:56.036383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82764a5d-aa9c-4323-b904-a9929d769288 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:56.036730 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1485215-c232-4fde-803b-457a40123f65 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:56.037070 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1485215-c232-4fde-803b-457a40123f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:56.037435 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1485215-c232-4fde-803b-457a40123f65 None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:56.038781 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1485215-c232-4fde-803b-457a40123f65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:56.039155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1485215-c232-4fde-803b-457a40123f65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:56.040559 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1485215-c232-4fde-803b-457a40123f65 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:56.062366 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 257/526] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:14:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:56.062366 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1485215-c232-4fde-803b-457a40123f65 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:56.121024 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:56.129955 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:56.149875 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 270/527] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:14:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:14:56.150958 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82764a5d-aa9c-4323-b904-a9929d769288 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:56.165601 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1485215-c232-4fde-803b-457a40123f65 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:56.202963 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7fdb54-1739-42e5-8356-5f5f6e55b310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:56.203867 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7fdb54-1739-42e5-8356-5f5f6e55b310 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:56.204317 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7fdb54-1739-42e5-8356-5f5f6e55b310 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:56.204771 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7fdb54-1739-42e5-8356-5f5f6e55b310 None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:56.205489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7fdb54-1739-42e5-8356-5f5f6e55b310 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:56.205793 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7fdb54-1739-42e5-8356-5f5f6e55b310 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:56.207227 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7fdb54-1739-42e5-8356-5f5f6e55b310 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:56.219213 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 258/528] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:14:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:56.219982 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7fdb54-1739-42e5-8356-5f5f6e55b310 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:56.271320 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82764a5d-aa9c-4323-b904-a9929d769288 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:56.306685 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7da5f1-15bf-4be6-920d-dbaccd5f86bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:56.307322 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7da5f1-15bf-4be6-920d-dbaccd5f86bc None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:56.307607 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7da5f1-15bf-4be6-920d-dbaccd5f86bc None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:56.307884 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7da5f1-15bf-4be6-920d-dbaccd5f86bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:56.323775 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7fdb54-1739-42e5-8356-5f5f6e55b310 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:56.345603 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:56.357326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691c1321-9e97-408d-a68a-c600120ff249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:56.357926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c1321-9e97-408d-a68a-c600120ff249 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:56.358206 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c1321-9e97-408d-a68a-c600120ff249 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:56.358480 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c1321-9e97-408d-a68a-c600120ff249 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:56.359199 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691c1321-9e97-408d-a68a-c600120ff249 None admin] RBAC: Adding query filter params (role.id=8ec32f702e0c447993ff294cbac536b1, scope.project.id=933820e2adbc473bbd57a4d59be75d72, user.id=87ece6128f9245f7b49f3e7a9faaed22) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:56.359504 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691c1321-9e97-408d-a68a-c600120ff249 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:56.360936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691c1321-9e97-408d-a68a-c600120ff249 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:56.365216 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:56.373229 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 259/529] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:14:56 2022] GET /identity/v3/role_assignments?user.id=87ece6128f9245f7b49f3e7a9faaed22&scope.project.id=933820e2adbc473bbd57a4d59be75d72&role.id=8ec32f702e0c447993ff294cbac536b1 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:56.373229 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691c1321-9e97-408d-a68a-c600120ff249 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:56.401956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7da5f1-15bf-4be6-920d-dbaccd5f86bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:56.404508 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7da5f1-15bf-4be6-920d-dbaccd5f86bc None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:56.487862 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691c1321-9e97-408d-a68a-c600120ff249 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:56.527427 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034dc913-69a3-4eb3-b1cf-c2658dd3976d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:56.528067 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034dc913-69a3-4eb3-b1cf-c2658dd3976d None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:56.528397 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034dc913-69a3-4eb3-b1cf-c2658dd3976d None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:56.528673 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034dc913-69a3-4eb3-b1cf-c2658dd3976d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:56.540544 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034dc913-69a3-4eb3-b1cf-c2658dd3976d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:56.542092 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034dc913-69a3-4eb3-b1cf-c2658dd3976d None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:56.553631 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 271/530] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:14:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:14:56.554512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7da5f1-15bf-4be6-920d-dbaccd5f86bc None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:56.558524 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-034dc913-69a3-4eb3-b1cf-c2658dd3976d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:14:56.558524 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:56.558524 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:56.558524 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:56.558524 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:56.558524 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:56.558524 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:56.558524 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:56.558524 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:56.558524 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:56.558524 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:56.558524 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:56.558524 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:14:56.558524 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:14:56.558524 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:14:56.558524 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:14:56.559435 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:56.559435 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:56.559435 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:56.559435 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:56.559435 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:56.559435 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:56.559435 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:56.559435 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:56.559435 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:56.559435 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:56.559435 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:56.559435 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:56.559435 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:56.559435 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:56.559435 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:56.559435 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:56.559435 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:14:56.560308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:14:56.560308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:14:56.560308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:14:56.562366 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 260/531] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:14:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:56.564141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034dc913-69a3-4eb3-b1cf-c2658dd3976d None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:56.618359 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:56.672448 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:56.690668 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7da5f1-15bf-4be6-920d-dbaccd5f86bc None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:56.695228 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034dc913-69a3-4eb3-b1cf-c2658dd3976d None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:56.726513 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69a55e-de82-49af-883a-9bed8f157212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:56.727504 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69a55e-de82-49af-883a-9bed8f157212 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:56.728043 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69a55e-de82-49af-883a-9bed8f157212 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:56.728506 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69a55e-de82-49af-883a-9bed8f157212 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:56.729524 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d69a55e-de82-49af-883a-9bed8f157212 None admin] RBAC: Adding query filter params (role.id=8ec32f702e0c447993ff294cbac536b1, scope.project.id=933820e2adbc473bbd57a4d59be75d72, user.id=11d86d2df1cb485592e8a5d7387131ee) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:56.730026 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d69a55e-de82-49af-883a-9bed8f157212 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:56.731755 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d69a55e-de82-49af-883a-9bed8f157212 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:56.734202 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa33d4f-a9b4-4738-9b97-1b2cce8e8efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:56.735183 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa33d4f-a9b4-4738-9b97-1b2cce8e8efe None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:56.735183 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa33d4f-a9b4-4738-9b97-1b2cce8e8efe None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:56.735480 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa33d4f-a9b4-4738-9b97-1b2cce8e8efe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:56.738751 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 272/532] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:14:56 2022] GET /identity/v3/role_assignments?user.id=11d86d2df1cb485592e8a5d7387131ee&scope.project.id=933820e2adbc473bbd57a4d59be75d72&role.id=8ec32f702e0c447993ff294cbac536b1 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:56.739834 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69a55e-de82-49af-883a-9bed8f157212 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:56.748210 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa33d4f-a9b4-4738-9b97-1b2cce8e8efe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:56.750407 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa33d4f-a9b4-4738-9b97-1b2cce8e8efe None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:56.769698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-9fa33d4f-a9b4-4738-9b97-1b2cce8e8efe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:14:56.769698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:56.769698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:56.769698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:56.769698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:56.769698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:56.769698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:56.769698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:56.769698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:56.769698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:56.769698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:56.769698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:56.769698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:14:56.769698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:14:56.769698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:14:56.769698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:14:56.770691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:56.770691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:56.770691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:56.770691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:56.770691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:56.770691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:56.770691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:56.770691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:56.770691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:56.770691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:56.770691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:56.770691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:56.770691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:56.770691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:56.770691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:56.770691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:56.770691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:14:56.771561 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:14:56.771561 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:14:56.771561 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:14:56.771561 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 261/533] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:14:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:56.782801 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa33d4f-a9b4-4738-9b97-1b2cce8e8efe None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:56.855265 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69a55e-de82-49af-883a-9bed8f157212 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:56.902705 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0965f-89d9-430b-ba85-d9a18b124c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:56.903761 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0965f-89d9-430b-ba85-d9a18b124c59 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:56.904236 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0965f-89d9-430b-ba85-d9a18b124c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:56.905710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0965f-89d9-430b-ba85-d9a18b124c59 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:56.906629 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0965f-89d9-430b-ba85-d9a18b124c59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:56.907327 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0965f-89d9-430b-ba85-d9a18b124c59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:56.910384 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0965f-89d9-430b-ba85-d9a18b124c59 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:56.936129 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 273/534] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:56.942244 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0965f-89d9-430b-ba85-d9a18b124c59 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:56.967215 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:57.026583 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:57.042375 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa33d4f-a9b4-4738-9b97-1b2cce8e8efe None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:57.094967 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0965f-89d9-430b-ba85-d9a18b124c59 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:57.121489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ae479-3c39-4096-81aa-9141e9bace72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:57.122112 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ae479-3c39-4096-81aa-9141e9bace72 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:57.122399 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ae479-3c39-4096-81aa-9141e9bace72 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:57.122740 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ae479-3c39-4096-81aa-9141e9bace72 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:57.123400 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ae479-3c39-4096-81aa-9141e9bace72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:57.123704 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ae479-3c39-4096-81aa-9141e9bace72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:57.125094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ae479-3c39-4096-81aa-9141e9bace72 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:57.153217 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 262/535] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:57.156745 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ae479-3c39-4096-81aa-9141e9bace72 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:57.178127 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429f4bf9-7f87-465c-a6bf-b1da5d973c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:57.178928 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f4bf9-7f87-465c-a6bf-b1da5d973c4c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:57.179214 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f4bf9-7f87-465c-a6bf-b1da5d973c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:57.179486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f4bf9-7f87-465c-a6bf-b1da5d973c4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:57.192941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429f4bf9-7f87-465c-a6bf-b1da5d973c4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:57.201297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429f4bf9-7f87-465c-a6bf-b1da5d973c4c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:57.222043 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-429f4bf9-7f87-465c-a6bf-b1da5d973c4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:14:57.222043 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:57.222043 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:57.222043 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:57.222043 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:57.222043 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:57.222043 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:57.222043 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:57.222043 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:57.222043 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:57.222043 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:57.222043 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:57.222043 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:14:57.222043 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:14:57.222043 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:14:57.223149 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:14:57.223149 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:57.223149 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:57.223149 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:57.223149 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:57.223149 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:57.223149 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:57.223149 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:57.223149 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:57.223149 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:57.223149 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:57.223149 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:57.223149 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:57.223149 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:57.223149 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:57.223149 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:57.223149 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:57.224172 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:14:57.224172 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:14:57.224172 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:14:57.224172 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:14:57.224172 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:14:57.224172 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:14:57.224172 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:14:57.224172 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:57.234745 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 274/536] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:14:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:57.240439 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429f4bf9-7f87-465c-a6bf-b1da5d973c4c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:57.372625 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ae479-3c39-4096-81aa-9141e9bace72 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:57.393655 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:57.478585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429f4bf9-7f87-465c-a6bf-b1da5d973c4c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:57.518064 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124601d-42ad-4342-ae54-a09d26e1068d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:57.518726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124601d-42ad-4342-ae54-a09d26e1068d None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:57.519009 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124601d-42ad-4342-ae54-a09d26e1068d None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:57.519288 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124601d-42ad-4342-ae54-a09d26e1068d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:57.527900 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f124601d-42ad-4342-ae54-a09d26e1068d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:57.529511 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f124601d-42ad-4342-ae54-a09d26e1068d None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:57.543376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-f124601d-42ad-4342-ae54-a09d26e1068d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:14:57.543376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:57.543376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:57.543376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:57.543376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:57.543376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:57.543376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:57.543376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:57.543376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:57.543376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:57.543376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:57.543376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:57.543376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:14:57.543376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:14:57.543376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:14:57.544498 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:14:57.544498 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:57.544498 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:57.544498 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:57.544498 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:57.544498 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:57.544498 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:57.544498 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:57.544498 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:57.544498 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:57.544498 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:57.544498 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:57.544498 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:57.544498 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:57.544498 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:57.544498 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:57.544498 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:57.545473 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:14:57.545473 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:14:57.545473 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:14:57.545473 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:14:57.545473 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:14:57.545473 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:14:57.545473 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:14:57.545473 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:14:57.545989 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 263/537] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:14:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:14:57.551975 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124601d-42ad-4342-ae54-a09d26e1068d None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:57.556812 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9120b7b-a635-455f-94d4-9d0ef0053e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:57.557479 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9120b7b-a635-455f-94d4-9d0ef0053e09 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:57.557761 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9120b7b-a635-455f-94d4-9d0ef0053e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:57.558035 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9120b7b-a635-455f-94d4-9d0ef0053e09 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:57.561700 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9120b7b-a635-455f-94d4-9d0ef0053e09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:57.562063 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9120b7b-a635-455f-94d4-9d0ef0053e09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:57.565064 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9120b7b-a635-455f-94d4-9d0ef0053e09 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:57.600694 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 275/538] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:14:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:57.606455 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9120b7b-a635-455f-94d4-9d0ef0053e09 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:57.609443 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:57.786414 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9120b7b-a635-455f-94d4-9d0ef0053e09 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:57.788072 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124601d-42ad-4342-ae54-a09d26e1068d None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:57.818657 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:57.823631 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a6414-913f-461a-8512-0c4f5ecef7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:57.824241 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a6414-913f-461a-8512-0c4f5ecef7d6 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:57.824661 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a6414-913f-461a-8512-0c4f5ecef7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:57.824944 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a6414-913f-461a-8512-0c4f5ecef7d6 None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:57.825657 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a6414-913f-461a-8512-0c4f5ecef7d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:57.825960 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a6414-913f-461a-8512-0c4f5ecef7d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:57.827492 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a6414-913f-461a-8512-0c4f5ecef7d6 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:57.828699 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f922c7-52c6-4f07-8bd4-fdf956927778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:57.829445 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f922c7-52c6-4f07-8bd4-fdf956927778 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:57.829726 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f922c7-52c6-4f07-8bd4-fdf956927778 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:57.830100 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f922c7-52c6-4f07-8bd4-fdf956927778 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:57.830898 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f922c7-52c6-4f07-8bd4-fdf956927778 None admin] RBAC: Adding query filter params (role.id=8ec32f702e0c447993ff294cbac536b1, scope.project.id=933820e2adbc473bbd57a4d59be75d72, user.id=ee5c6e51f2694bf5ae7ea097cf33fb3b) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:57.831207 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f922c7-52c6-4f07-8bd4-fdf956927778 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:57.832613 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f922c7-52c6-4f07-8bd4-fdf956927778 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:57.838929 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 276/539] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:14:57 2022] GET /identity/v3/role_assignments?user.id=ee5c6e51f2694bf5ae7ea097cf33fb3b&scope.project.id=933820e2adbc473bbd57a4d59be75d72&role.id=8ec32f702e0c447993ff294cbac536b1 => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:57.840195 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4ff0f-fe84-4973-b451-e0f961025d1b None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:57.840308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 264/540] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:14:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:57.840476 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4ff0f-fe84-4973-b451-e0f961025d1b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:57.840754 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4ff0f-fe84-4973-b451-e0f961025d1b None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:57.842218 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 277/541] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:57.846055 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a6414-913f-461a-8512-0c4f5ecef7d6 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:57.859774 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb2ba2-e706-429c-9b57-e9fe37335702 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:57.860087 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb2ba2-e706-429c-9b57-e9fe37335702 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:57.860331 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb2ba2-e706-429c-9b57-e9fe37335702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:57.896543 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-09eb2ba2-e706-429c-9b57-e9fe37335702 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:57.897860 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-09eb2ba2-e706-429c-9b57-e9fe37335702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:58.023080 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 278/542] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:58.026715 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:58.061898 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a6414-913f-461a-8512-0c4f5ecef7d6 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:58.097767 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7a1dd-f726-4db0-8436-53160b69d86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:58.098657 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7a1dd-f726-4db0-8436-53160b69d86c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:58.099137 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7a1dd-f726-4db0-8436-53160b69d86c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:58.099606 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7a1dd-f726-4db0-8436-53160b69d86c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:58.100579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7a1dd-f726-4db0-8436-53160b69d86c None admin] RBAC: Adding query filter params (role.id=8ec32f702e0c447993ff294cbac536b1, scope.project.id=933820e2adbc473bbd57a4d59be75d72, user.id=5298486310004b63b51116dbc747e31d) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:58.101104 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7a1dd-f726-4db0-8436-53160b69d86c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:58.102741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7a1dd-f726-4db0-8436-53160b69d86c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:58.108692 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 265/543] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:14:57 2022] GET /identity/v3/role_assignments?user.id=5298486310004b63b51116dbc747e31d&scope.project.id=933820e2adbc473bbd57a4d59be75d72&role.id=8ec32f702e0c447993ff294cbac536b1 => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:58.347887 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb2ba2-e706-429c-9b57-e9fe37335702 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:58.461411 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb2ba2-e706-429c-9b57-e9fe37335702 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:58.495881 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:58.509486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54b1cb-2c16-49df-823f-bb631210641a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:58.510118 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54b1cb-2c16-49df-823f-bb631210641a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:58.510396 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54b1cb-2c16-49df-823f-bb631210641a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:58.510670 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54b1cb-2c16-49df-823f-bb631210641a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:58.530164 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e54b1cb-2c16-49df-823f-bb631210641a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:58.535080 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-3e54b1cb-2c16-49df-823f-bb631210641a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:14:58.535080 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:58.535080 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:58.535080 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:58.535080 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:58.535080 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:58.535080 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:58.535080 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:58.535080 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:58.535080 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:58.535080 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:58.535080 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:58.535080 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:58.535080 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:58.535080 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:58.535080 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:58.535972 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:58.535972 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:58.535972 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:58.535972 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:58.535972 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:58.535972 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:58.535972 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:58.535972 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:58.535972 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:58.535972 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:58.535972 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:58.535972 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:58.535972 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:58.535972 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:58.535972 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:58.535972 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:58.536783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:58.536783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:58.536783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:58.536783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:58.536783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:58.536783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:58.536783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:58.536783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:58.536783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:58.536783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:58.536783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:14:58.536783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:58.536783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 279/544] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Sep 9 20:14:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:58.543932 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7a1dd-f726-4db0-8436-53160b69d86c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:58.698190 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7a1dd-f726-4db0-8436-53160b69d86c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:58.747368 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445c953-8dd1-4359-9fea-ba9fe27552b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:58.747984 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445c953-8dd1-4359-9fea-ba9fe27552b5 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:58.748263 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445c953-8dd1-4359-9fea-ba9fe27552b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:58.748533 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445c953-8dd1-4359-9fea-ba9fe27552b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:58.749208 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445c953-8dd1-4359-9fea-ba9fe27552b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:58.749536 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445c953-8dd1-4359-9fea-ba9fe27552b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:58.750894 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445c953-8dd1-4359-9fea-ba9fe27552b5 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:58.753168 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:58.760842 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 266/545] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Sep 9 20:14:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:58.766346 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54b1cb-2c16-49df-823f-bb631210641a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:58.828267 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae807d-bcd3-4e9d-be30-05a0878c4e3f None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:58.828717 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae807d-bcd3-4e9d-be30-05a0878c4e3f None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:58.829111 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae807d-bcd3-4e9d-be30-05a0878c4e3f None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:58.830856 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 267/546] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:58.836026 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b186fd-42b9-4854-9a9a-76b54c92d786 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:58.836331 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b186fd-42b9-4854-9a9a-76b54c92d786 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:58.836575 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b186fd-42b9-4854-9a9a-76b54c92d786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:58.856895 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-a9b186fd-42b9-4854-9a9a-76b54c92d786 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:58.858185 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a9b186fd-42b9-4854-9a9a-76b54c92d786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:58.941112 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54b1cb-2c16-49df-823f-bb631210641a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:58.964815 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 268/547] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:14:58.992287 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b75149-a3b1-4f65-a2a2-8e6d0ab95f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:58.992927 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b75149-a3b1-4f65-a2a2-8e6d0ab95f48 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:58.993274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b75149-a3b1-4f65-a2a2-8e6d0ab95f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:58.993546 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b75149-a3b1-4f65-a2a2-8e6d0ab95f48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:59.004381 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b75149-a3b1-4f65-a2a2-8e6d0ab95f48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:59.005917 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b75149-a3b1-4f65-a2a2-8e6d0ab95f48 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:59.020923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-09b75149-a3b1-4f65-a2a2-8e6d0ab95f48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:14:59.020923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:59.020923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:59.020923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:59.020923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:59.020923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:59.020923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:59.020923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:59.020923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:59.020923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:59.020923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:59.020923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:59.020923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:14:59.020923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:14:59.020923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:14:59.020923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:14:59.021891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:59.021891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:59.021891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:59.021891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:59.021891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:59.021891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:59.021891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:59.021891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:59.021891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:59.021891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:59.021891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:59.021891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:59.021891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:59.021891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:59.021891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:59.021891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:59.021891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:14:59.023291 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:14:59.023291 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:14:59.023291 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:59.023291 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 280/548] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:14:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:59.029259 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b186fd-42b9-4854-9a9a-76b54c92d786 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:59.167001 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b186fd-42b9-4854-9a9a-76b54c92d786 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:59.202809 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:59.212762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad994df-a393-4463-b3f5-6433e9b40a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:59.213414 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad994df-a393-4463-b3f5-6433e9b40a2a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:59.213691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad994df-a393-4463-b3f5-6433e9b40a2a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:59.213956 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad994df-a393-4463-b3f5-6433e9b40a2a None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:59.214597 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad994df-a393-4463-b3f5-6433e9b40a2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:59.214893 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad994df-a393-4463-b3f5-6433e9b40a2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:59.216245 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad994df-a393-4463-b3f5-6433e9b40a2a None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:59.234220 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 269/549] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:59.239880 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b75149-a3b1-4f65-a2a2-8e6d0ab95f48 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:59.251704 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad994df-a393-4463-b3f5-6433e9b40a2a None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:59.297925 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:59.425199 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b75149-a3b1-4f65-a2a2-8e6d0ab95f48 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:59.441142 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad994df-a393-4463-b3f5-6433e9b40a2a None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:59.460746 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6399b07-5442-417e-a3f1-f7e0fbac9e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:59.461435 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6399b07-5442-417e-a3f1-f7e0fbac9e30 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:59.461716 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6399b07-5442-417e-a3f1-f7e0fbac9e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:59.461988 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6399b07-5442-417e-a3f1-f7e0fbac9e30 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:59.471929 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6399b07-5442-417e-a3f1-f7e0fbac9e30 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:59.473506 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6399b07-5442-417e-a3f1-f7e0fbac9e30 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:59.486395 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6483719-df60-4125-8dcf-98aa311fb056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:59.486996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-b6399b07-5442-417e-a3f1-f7e0fbac9e30 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 20:14:59.486996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:59.486996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:59.486996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:59.486996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:59.486996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:59.486996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:59.486996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:59.486996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:59.486996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:59.486996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:59.486996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:59.486996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:14:59.486996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:14:59.486996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:14:59.486996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:14:59.488083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:59.488083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:59.488083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:14:59.488083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:14:59.488083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:14:59.488083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:14:59.488083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:59.488083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:59.488083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:59.488083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:59.488083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:59.488083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:59.488083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:59.488083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:59.488083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:59.488083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:59.488083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:59.488083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:59.488998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:59.488998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:59.488998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:14:59.488998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:14:59.488998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:14:59.488998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:14:59.488998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:14:59.488998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:14:59.488998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 20:14:59.488998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:14:59.490759 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6483719-df60-4125-8dcf-98aa311fb056 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:59.491400 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6483719-df60-4125-8dcf-98aa311fb056 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:59.491893 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6483719-df60-4125-8dcf-98aa311fb056 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:59.492575 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 281/550] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:14:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:14:59.493851 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51150b-7800-493a-a2f2-e235480bbd68 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:59.494136 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51150b-7800-493a-a2f2-e235480bbd68 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:59.494412 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51150b-7800-493a-a2f2-e235480bbd68 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:59.496352 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 282/551] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:14:59.497220 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f51150b-7800-493a-a2f2-e235480bbd68 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:59.505349 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6483719-df60-4125-8dcf-98aa311fb056 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:14:59.511822 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-b6483719-df60-4125-8dcf-98aa311fb056 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:14:59.511822 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:59.511822 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:59.511822 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:59.511822 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:59.511822 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:59.511822 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:59.511822 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:59.511822 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:59.511822 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:59.511822 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:59.511822 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:59.511822 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:14:59.511822 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:14:59.511822 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:14:59.511822 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:14:59.512752 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:14:59.512752 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:14:59.512752 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:14:59.512752 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:14:59.512752 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:59.512752 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:59.512752 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:59.512752 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:59.512752 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:59.512752 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:59.512752 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:59.512752 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:59.512752 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:59.512752 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:59.512752 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:59.512752 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:59.513774 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:59.513774 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:59.513774 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:59.513774 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:59.513774 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:14:59.513774 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:14:59.513774 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:14:59.513774 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:14:59.513774 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:14:59.513774 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:14:59.513774 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:14:59.513774 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:14:59.516422 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 270/552] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Sep 9 20:14:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:14:59.517812 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4742ab-093d-4c87-b1b1-d188bad65ece None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:59.518295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4742ab-093d-4c87-b1b1-d188bad65ece None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:59.518764 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4742ab-093d-4c87-b1b1-d188bad65ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:59.538855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-4e4742ab-093d-4c87-b1b1-d188bad65ece None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:14:59.540651 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-4e4742ab-093d-4c87-b1b1-d188bad65ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:14:59.587802 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:59.603818 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:59.611561 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f51150b-7800-493a-a2f2-e235480bbd68 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:59.616396 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 271/553] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:14:59.617723 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4742ab-093d-4c87-b1b1-d188bad65ece None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:59.649753 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b87caff-5fd3-4315-b33e-729818efb0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:59.650546 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87caff-5fd3-4315-b33e-729818efb0bb None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:59.651026 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87caff-5fd3-4315-b33e-729818efb0bb None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:59.651414 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87caff-5fd3-4315-b33e-729818efb0bb None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:59.652237 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b87caff-5fd3-4315-b33e-729818efb0bb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:59.652641 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b87caff-5fd3-4315-b33e-729818efb0bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:59.654289 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b87caff-5fd3-4315-b33e-729818efb0bb None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:59.686702 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 283/554] 10.222.0.26 () {56 vars in 1135 bytes} [Fri Sep 9 20:14:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:14:59.689725 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b87caff-5fd3-4315-b33e-729818efb0bb None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:59.724688 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4742ab-093d-4c87-b1b1-d188bad65ece None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:59.778524 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1220ecd5-db0e-42a3-a8da-9bdbea42c9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:59.779612 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220ecd5-db0e-42a3-a8da-9bdbea42c9e2 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:59.780093 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220ecd5-db0e-42a3-a8da-9bdbea42c9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:59.780613 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220ecd5-db0e-42a3-a8da-9bdbea42c9e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:59.781715 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1220ecd5-db0e-42a3-a8da-9bdbea42c9e2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:14:59.782328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1220ecd5-db0e-42a3-a8da-9bdbea42c9e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:59.784127 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1220ecd5-db0e-42a3-a8da-9bdbea42c9e2 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:59.793766 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 272/555] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Sep 9 20:14:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:14:59.800146 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1220ecd5-db0e-42a3-a8da-9bdbea42c9e2 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:14:59.824637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b87caff-5fd3-4315-b33e-729818efb0bb None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:59.847267 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:59.875752 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61443f-6bfa-4919-99c4-96b43d3a85a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:59.876369 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61443f-6bfa-4919-99c4-96b43d3a85a7 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:59.876653 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61443f-6bfa-4919-99c4-96b43d3a85a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:59.876935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61443f-6bfa-4919-99c4-96b43d3a85a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:59.912657 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1220ecd5-db0e-42a3-a8da-9bdbea42c9e2 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:14:59.915307 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:14:59.951546 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef1423-be0c-45e6-9d42-e252e1e3e1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:14:59.952604 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef1423-be0c-45e6-9d42-e252e1e3e1ff None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:14:59.953110 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef1423-be0c-45e6-9d42-e252e1e3e1ff None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:14:59.953661 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef1423-be0c-45e6-9d42-e252e1e3e1ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:14:59.965971 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ef1423-be0c-45e6-9d42-e252e1e3e1ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:59.967823 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ef1423-be0c-45e6-9d42-e252e1e3e1ff None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:59.970139 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61443f-6bfa-4919-99c4-96b43d3a85a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:14:59.971668 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61443f-6bfa-4919-99c4-96b43d3a85a7 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:14:59.983969 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-20ef1423-be0c-45e6-9d42-e252e1e3e1ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:14:59.983969 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:14:59.983969 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:14:59.983969 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:14:59.983969 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:14:59.983969 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:14:59.983969 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:14:59.983969 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:14:59.983969 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:14:59.983969 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:14:59.983969 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:14:59.983969 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:14:59.983969 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:14:59.983969 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:14:59.983969 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:14:59.983969 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:14:59.984926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:14:59.984926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:14:59.984926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:14:59.984926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:14:59.984926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:14:59.984926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:14:59.984926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:14:59.984926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:14:59.984926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:14:59.984926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:14:59.984926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:14:59.984926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:14:59.984926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:14:59.984926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:14:59.984926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:14:59.984926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:14:59.984926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:14:59.985943 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:14:59.985943 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:14:59.985943 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:14:59.987705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 273/556] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:14:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:14:59.993545 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef1423-be0c-45e6-9d42-e252e1e3e1ff None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:00.089941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 284/557] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:14:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:15:00.091344 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:00.091785 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61443f-6bfa-4919-99c4-96b43d3a85a7 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:00.103518 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef1423-be0c-45e6-9d42-e252e1e3e1ff None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:00.141789 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef05b8f-0a6b-41c0-923b-614a92166b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:00.142500 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef05b8f-0a6b-41c0-923b-614a92166b6e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:00.142779 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef05b8f-0a6b-41c0-923b-614a92166b6e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:00.143053 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef05b8f-0a6b-41c0-923b-614a92166b6e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:00.154603 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef05b8f-0a6b-41c0-923b-614a92166b6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:15:00.155071 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:00.161409 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-8ef05b8f-0a6b-41c0-923b-614a92166b6e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:15:00.161409 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:00.161409 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:00.161409 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:00.161409 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:00.161409 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:00.161409 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:00.161409 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:00.161409 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:00.161409 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:00.161409 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:00.161409 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:00.161409 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:15:00.161409 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:15:00.161409 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:15:00.161409 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:15:00.163493 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:15:00.163493 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:15:00.163493 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:15:00.163493 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:15:00.163493 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:00.163493 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:00.163493 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:00.163493 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:00.163493 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:00.163493 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:00.163493 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:00.163493 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:00.163493 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:00.163493 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:00.163493 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:00.163493 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:00.164343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:00.164343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:00.164343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:00.164343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:00.164343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:15:00.164343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:15:00.164343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:15:00.164343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:15:00.164343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:15:00.164343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:15:00.164343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:15:00.164343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:00.164343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 274/558] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Sep 9 20:14:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:00.165096 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef05b8f-0a6b-41c0-923b-614a92166b6e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:00.217042 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61443f-6bfa-4919-99c4-96b43d3a85a7 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:00.258710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb34ed5-4f2d-4c59-a5df-d3e5beda7435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:00.259917 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb34ed5-4f2d-4c59-a5df-d3e5beda7435 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:00.260344 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb34ed5-4f2d-4c59-a5df-d3e5beda7435 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:00.260766 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb34ed5-4f2d-4c59-a5df-d3e5beda7435 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:00.261592 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb34ed5-4f2d-4c59-a5df-d3e5beda7435 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:00.262033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb34ed5-4f2d-4c59-a5df-d3e5beda7435 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:00.264040 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb34ed5-4f2d-4c59-a5df-d3e5beda7435 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:00.275914 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef05b8f-0a6b-41c0-923b-614a92166b6e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:00.278260 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 285/559] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:00.279239 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb34ed5-4f2d-4c59-a5df-d3e5beda7435 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:00.318076 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbfd75a-4a1a-4eca-97cd-b301edc3b2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:00.319581 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbfd75a-4a1a-4eca-97cd-b301edc3b2df None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:00.319961 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbfd75a-4a1a-4eca-97cd-b301edc3b2df None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:00.320357 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbfd75a-4a1a-4eca-97cd-b301edc3b2df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:00.334154 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbfd75a-4a1a-4eca-97cd-b301edc3b2df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:00.336314 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbfd75a-4a1a-4eca-97cd-b301edc3b2df None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:00.354811 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-ffbfd75a-4a1a-4eca-97cd-b301edc3b2df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:00.354811 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:00.354811 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:00.354811 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:00.354811 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:00.354811 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:00.354811 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:00.354811 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:00.354811 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:00.354811 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:00.354811 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:00.354811 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:00.354811 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:00.354811 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:00.354811 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:00.354811 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:00.355907 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:00.355907 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:00.355907 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:00.355907 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:00.355907 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:00.355907 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:00.355907 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:00.355907 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:00.355907 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:00.355907 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:00.355907 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:00.355907 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:00.355907 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:00.355907 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:00.355907 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:00.355907 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:00.355907 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:00.356798 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:00.356798 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:00.356798 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:00.359132 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 275/560] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:00.360364 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbfd75a-4a1a-4eca-97cd-b301edc3b2df None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:00.365122 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:00.398869 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb34ed5-4f2d-4c59-a5df-d3e5beda7435 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:00.415717 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:00.440403 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1bde5c-02b8-4874-91a4-426806159297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:00.441220 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bde5c-02b8-4874-91a4-426806159297 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:00.441627 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bde5c-02b8-4874-91a4-426806159297 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:00.442003 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bde5c-02b8-4874-91a4-426806159297 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:00.443029 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1bde5c-02b8-4874-91a4-426806159297 None admin] RBAC: Adding query filter params (name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:00.443806 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1bde5c-02b8-4874-91a4-426806159297 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:00.445621 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1bde5c-02b8-4874-91a4-426806159297 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:00.460820 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 286/561] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Sep 9 20:15:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:00.460820 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1bde5c-02b8-4874-91a4-426806159297 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:00.496276 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbfd75a-4a1a-4eca-97cd-b301edc3b2df None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:00.535348 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea0318-6350-4ac8-bf77-3afc204ccafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:00.536052 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea0318-6350-4ac8-bf77-3afc204ccafe None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:00.536502 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea0318-6350-4ac8-bf77-3afc204ccafe None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:00.536801 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea0318-6350-4ac8-bf77-3afc204ccafe None admin] PATH_INFO: `/v3/users/nova` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:00.545371 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ea0318-6350-4ac8-bf77-3afc204ccafe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:00.546984 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ea0318-6350-4ac8-bf77-3afc204ccafe None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:00.561442 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-a3ea0318-6350-4ac8-bf77-3afc204ccafe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:15:00.561442 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:00.561442 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:00.561442 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:00.561442 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:00.561442 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:00.561442 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:00.561442 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:00.561442 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:00.561442 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:00.561442 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:00.561442 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:00.561442 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:15:00.561442 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:15:00.561442 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:15:00.561442 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:15:00.562632 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:00.562632 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:00.562632 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:15:00.562632 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:00.562632 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:15:00.562632 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:00.562632 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:00.562632 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:00.562632 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:00.562632 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:00.562632 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:00.562632 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:00.562632 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:00.562632 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:00.562632 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:00.562632 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:00.562632 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:00.562632 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:00.563625 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:00.563625 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:00.563625 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:15:00.563625 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:15:00.563625 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:15:00.563625 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:15:00.563625 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:15:00.563625 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:15:00.563625 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:15:00.563625 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:00.568495 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 276/562] 10.222.0.26 () {56 vars in 1093 bytes} [Fri Sep 9 20:15:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:00.569739 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea0318-6350-4ac8-bf77-3afc204ccafe None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:00.588202 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1bde5c-02b8-4874-91a4-426806159297 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:00.627844 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33427ba9-d18c-4e81-86b8-da938337ab32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:00.629570 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33427ba9-d18c-4e81-86b8-da938337ab32 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:00.630320 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33427ba9-d18c-4e81-86b8-da938337ab32 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:00.630770 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33427ba9-d18c-4e81-86b8-da938337ab32 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:00.632414 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33427ba9-d18c-4e81-86b8-da938337ab32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:00.633271 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33427ba9-d18c-4e81-86b8-da938337ab32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:00.635325 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33427ba9-d18c-4e81-86b8-da938337ab32 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:00.639074 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:00.649478 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 287/563] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:00.654553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33427ba9-d18c-4e81-86b8-da938337ab32 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:00.680658 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea0318-6350-4ac8-bf77-3afc204ccafe None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:00.696762 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:00.719773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5da9a9-1fcd-4d63-be05-da7e02e310cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:00.720427 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5da9a9-1fcd-4d63-be05-da7e02e310cd None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:00.720701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5da9a9-1fcd-4d63-be05-da7e02e310cd None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:00.720955 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5da9a9-1fcd-4d63-be05-da7e02e310cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:00.731939 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5da9a9-1fcd-4d63-be05-da7e02e310cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:00.733431 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5da9a9-1fcd-4d63-be05-da7e02e310cd None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:00.747853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-9c5da9a9-1fcd-4d63-be05-da7e02e310cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:00.747853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:00.747853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:00.747853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:00.747853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:00.747853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:00.747853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:00.747853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:00.747853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:00.747853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:00.747853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:00.747853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:00.747853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:00.747853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:00.747853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:00.747853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:00.749014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:00.749014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:00.749014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:00.749014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:00.749014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:00.749014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:00.749014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:00.749014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:00.749014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:00.749014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:00.749014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:00.749014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:00.749014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:00.749014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:00.749014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:00.749014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:00.749014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:00.749988 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:00.749988 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:00.749988 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:00.749988 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 277/564] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:00.750225 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6bb4b8-4de1-4f21-b6b1-73a6776d4d9d None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:00.750450 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6bb4b8-4de1-4f21-b6b1-73a6776d4d9d None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:00.750714 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6bb4b8-4de1-4f21-b6b1-73a6776d4d9d None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:00.756316 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 278/565] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:00.757207 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6bb4b8-4de1-4f21-b6b1-73a6776d4d9d None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:00.776999 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33427ba9-d18c-4e81-86b8-da938337ab32 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:00.814624 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37695ee-175c-4751-a02d-cda994f0db65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:00.815155 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37695ee-175c-4751-a02d-cda994f0db65 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:00.815593 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37695ee-175c-4751-a02d-cda994f0db65 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:00.815866 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37695ee-175c-4751-a02d-cda994f0db65 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:00.816922 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37695ee-175c-4751-a02d-cda994f0db65 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:00.817229 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37695ee-175c-4751-a02d-cda994f0db65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:00.818671 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37695ee-175c-4751-a02d-cda994f0db65 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:00.836548 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 288/566] 10.222.0.26 () {56 vars in 1133 bytes} [Fri Sep 9 20:15:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:00.837152 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37695ee-175c-4751-a02d-cda994f0db65 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:00.859833 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6bb4b8-4de1-4f21-b6b1-73a6776d4d9d None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:00.875232 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:00.894138 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220423cb-7b91-4c9c-a8c9-4ca5387d6694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:00.894760 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220423cb-7b91-4c9c-a8c9-4ca5387d6694 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:00.895037 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220423cb-7b91-4c9c-a8c9-4ca5387d6694 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:00.895312 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220423cb-7b91-4c9c-a8c9-4ca5387d6694 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:00.902393 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220423cb-7b91-4c9c-a8c9-4ca5387d6694 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:00.903841 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220423cb-7b91-4c9c-a8c9-4ca5387d6694 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:00.915682 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-220423cb-7b91-4c9c-a8c9-4ca5387d6694 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:00.915682 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:00.915682 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:00.915682 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:00.915682 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:00.915682 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:00.915682 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:00.915682 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:00.915682 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:00.915682 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:00.915682 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:00.915682 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:00.915682 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:15:00.915682 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:15:00.915682 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:15:00.916738 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:15:00.916738 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:00.916738 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:00.916738 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:00.916738 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:00.916738 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:00.916738 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:00.916738 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:00.916738 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:00.916738 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:00.916738 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:00.916738 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:00.916738 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:00.916738 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:00.916738 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:00.916738 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:00.916738 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:00.917876 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:15:00.917876 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:15:00.917876 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:15:00.917876 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:15:00.917876 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:15:00.917876 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:15:00.917876 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:00.917876 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:00.917876 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 279/567] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:15:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:00.917876 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbec85-601d-4482-a130-14a5b693c61b None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:00.918537 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbec85-601d-4482-a130-14a5b693c61b None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:00.918537 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbec85-601d-4482-a130-14a5b693c61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:00.933515 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:00.935584 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-3acbec85-601d-4482-a130-14a5b693c61b None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:00.937009 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-3acbec85-601d-4482-a130-14a5b693c61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:00.942692 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37695ee-175c-4751-a02d-cda994f0db65 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:00.976231 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06c9da-25f9-4944-942a-19eb348b6eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:00.976838 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06c9da-25f9-4944-942a-19eb348b6eba None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:00.977120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06c9da-25f9-4944-942a-19eb348b6eba None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:00.977433 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06c9da-25f9-4944-942a-19eb348b6eba None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:00.978072 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e06c9da-25f9-4944-942a-19eb348b6eba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:00.978373 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e06c9da-25f9-4944-942a-19eb348b6eba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:00.979712 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e06c9da-25f9-4944-942a-19eb348b6eba None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:00.990503 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 289/568] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:00.995094 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb7bbb-6264-4435-b07c-838c5069e920 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:00.995094 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb7bbb-6264-4435-b07c-838c5069e920 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:00.995094 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb7bbb-6264-4435-b07c-838c5069e920 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:00.995094 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 290/569] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:00.995094 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb7bbb-6264-4435-b07c-838c5069e920 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:01.008053 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 280/570] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:15:01.008904 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acbec85-601d-4482-a130-14a5b693c61b None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:01.110050 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb7bbb-6264-4435-b07c-838c5069e920 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:01.114423 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acbec85-601d-4482-a130-14a5b693c61b None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:01.121759 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:01.152091 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673de10-893d-4db2-8674-5a783e8ec797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:01.152916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673de10-893d-4db2-8674-5a783e8ec797 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:01.153162 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd5d2f-3214-4d5f-adb2-f2165a1fbb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:01.153546 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673de10-893d-4db2-8674-5a783e8ec797 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:01.153974 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd5d2f-3214-4d5f-adb2-f2165a1fbb17 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:01.154113 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673de10-893d-4db2-8674-5a783e8ec797 None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:01.154274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd5d2f-3214-4d5f-adb2-f2165a1fbb17 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:01.154765 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd5d2f-3214-4d5f-adb2-f2165a1fbb17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:01.155209 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6673de10-893d-4db2-8674-5a783e8ec797 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:01.155648 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6673de10-893d-4db2-8674-5a783e8ec797 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:01.157217 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6673de10-893d-4db2-8674-5a783e8ec797 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:01.167556 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cd5d2f-3214-4d5f-adb2-f2165a1fbb17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:01.168989 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cd5d2f-3214-4d5f-adb2-f2165a1fbb17 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:01.169289 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 281/571] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:15:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:01.175242 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0cc954-55cc-4353-b267-4bd18cf99531 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:01.175528 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0cc954-55cc-4353-b267-4bd18cf99531 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:01.175814 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0cc954-55cc-4353-b267-4bd18cf99531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:01.184883 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-e0cd5d2f-3214-4d5f-adb2-f2165a1fbb17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:01.184883 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:01.184883 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:01.184883 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:01.184883 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:01.184883 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:01.184883 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:01.184883 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:01.184883 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:01.184883 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:01.184883 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:01.184883 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:01.184883 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:01.184883 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:01.184883 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:01.184883 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:01.185864 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:01.185864 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:01.185864 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:01.185864 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:01.185864 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:01.185864 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:01.185864 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:01.185864 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:01.185864 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:01.185864 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:01.185864 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:01.185864 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:01.185864 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:01.185864 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:01.185864 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:01.185864 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:01.185864 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:01.186832 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:01.186832 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:01.186832 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:01.190730 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 291/572] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:01.191246 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd5d2f-3214-4d5f-adb2-f2165a1fbb17 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:01.195153 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-1a0cc954-55cc-4353-b267-4bd18cf99531 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:01.196323 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-1a0cc954-55cc-4353-b267-4bd18cf99531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:01.196447 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:01.267683 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 282/573] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:15:01.269349 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0cc954-55cc-4353-b267-4bd18cf99531 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:01.297805 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd5d2f-3214-4d5f-adb2-f2165a1fbb17 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:01.332688 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66389a78-9aaa-473e-b038-f51c7980ec53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:01.333762 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66389a78-9aaa-473e-b038-f51c7980ec53 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:01.334049 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66389a78-9aaa-473e-b038-f51c7980ec53 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:01.334325 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66389a78-9aaa-473e-b038-f51c7980ec53 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:01.343065 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66389a78-9aaa-473e-b038-f51c7980ec53 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:01.345057 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66389a78-9aaa-473e-b038-f51c7980ec53 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:01.357654 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-66389a78-9aaa-473e-b038-f51c7980ec53 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 20:15:01.357654 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:01.357654 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:01.357654 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:01.357654 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:01.357654 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:01.357654 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:01.357654 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:01.357654 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:01.357654 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:01.357654 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:01.357654 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:01.357654 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:15:01.357654 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:15:01.357654 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:15:01.357654 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:15:01.358618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:01.358618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:01.358618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:15:01.358618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:01.358618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:15:01.358618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:01.358618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:01.358618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:01.358618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:01.358618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:01.358618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:01.358618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:01.358618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:01.358618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:01.358618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:01.358618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:01.358618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:01.358618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:01.359629 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:01.359629 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:01.359629 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:15:01.359629 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:15:01.359629 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:15:01.359629 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:15:01.359629 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:15:01.359629 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:15:01.359629 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 20:15:01.359629 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:01.359629 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 292/574] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:15:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:01.360873 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66389a78-9aaa-473e-b038-f51c7980ec53 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:01.376608 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:01.385642 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0cc954-55cc-4353-b267-4bd18cf99531 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:01.426510 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:01.427858 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f35b22d-3685-4230-b105-6d72138a4bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:01.428249 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35b22d-3685-4230-b105-6d72138a4bda None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:01.428441 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35b22d-3685-4230-b105-6d72138a4bda None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:01.428849 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35b22d-3685-4230-b105-6d72138a4bda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:01.429593 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f35b22d-3685-4230-b105-6d72138a4bda None admin] RBAC: Adding query filter params (role.id=8ec32f702e0c447993ff294cbac536b1, scope.project.id=933820e2adbc473bbd57a4d59be75d72, user.id=fcb62d06f7e94e9295c54504ddabb1d4) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:01.429882 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f35b22d-3685-4230-b105-6d72138a4bda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:01.431302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f35b22d-3685-4230-b105-6d72138a4bda None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:01.436547 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 283/575] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:15:01 2022] GET /identity/v3/role_assignments?user.id=fcb62d06f7e94e9295c54504ddabb1d4&scope.project.id=933820e2adbc473bbd57a4d59be75d72&role.id=8ec32f702e0c447993ff294cbac536b1 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:01.442422 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f35b22d-3685-4230-b105-6d72138a4bda None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:01.477110 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66389a78-9aaa-473e-b038-f51c7980ec53 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:01.524236 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9531756-5004-4a00-97fa-6da7bf809e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:01.525481 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9531756-5004-4a00-97fa-6da7bf809e5d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:01.525947 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9531756-5004-4a00-97fa-6da7bf809e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:01.526371 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9531756-5004-4a00-97fa-6da7bf809e5d None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:01.527190 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9531756-5004-4a00-97fa-6da7bf809e5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:01.527669 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9531756-5004-4a00-97fa-6da7bf809e5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:01.529218 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9531756-5004-4a00-97fa-6da7bf809e5d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:01.549327 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 293/576] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:01.549327 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9531756-5004-4a00-97fa-6da7bf809e5d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:01.578491 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f35b22d-3685-4230-b105-6d72138a4bda None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:01.614684 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68d23b-7d0c-4756-9e24-a9fad825ab31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:01.615338 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68d23b-7d0c-4756-9e24-a9fad825ab31 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:01.615786 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68d23b-7d0c-4756-9e24-a9fad825ab31 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:01.616148 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68d23b-7d0c-4756-9e24-a9fad825ab31 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:01.635825 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d68d23b-7d0c-4756-9e24-a9fad825ab31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:15:01.641041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-2d68d23b-7d0c-4756-9e24-a9fad825ab31 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:15:01.641041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:01.641041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:01.641041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:01.641041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:01.641041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:01.641041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:01.641041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:01.641041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:01.641041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:01.641041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:01.641041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:01.641041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:15:01.641041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:15:01.641041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:15:01.641041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:15:01.642249 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:15:01.642249 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:15:01.642249 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:15:01.642249 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:15:01.642249 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:01.642249 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:01.642249 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:01.642249 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:01.642249 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:01.642249 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:01.642249 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:01.642249 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:01.642249 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:01.642249 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:01.642249 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:01.642249 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:01.643117 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:01.643117 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:01.643117 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:01.643117 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:01.643117 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:15:01.643117 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:15:01.643117 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:15:01.643117 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:15:01.643117 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:15:01.643117 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:15:01.643117 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:15:01.643117 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:01.643117 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 284/577] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Sep 9 20:15:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:01.643117 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68d23b-7d0c-4756-9e24-a9fad825ab31 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:01.665920 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9531756-5004-4a00-97fa-6da7bf809e5d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:01.668657 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:01.696604 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:01.709680 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a96486-a3bf-4ff2-b462-e597633d2bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:01.710294 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a96486-a3bf-4ff2-b462-e597633d2bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:01.710574 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a96486-a3bf-4ff2-b462-e597633d2bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:01.710846 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a96486-a3bf-4ff2-b462-e597633d2bc7 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:01.711565 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a96486-a3bf-4ff2-b462-e597633d2bc7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:01.711867 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a96486-a3bf-4ff2-b462-e597633d2bc7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:01.713946 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a96486-a3bf-4ff2-b462-e597633d2bc7 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:01.732467 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 294/578] 10.222.0.26 () {56 vars in 1137 bytes} [Fri Sep 9 20:15:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:01.734176 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a96486-a3bf-4ff2-b462-e597633d2bc7 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:01.757657 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68d23b-7d0c-4756-9e24-a9fad825ab31 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:01.791860 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f896026f-e6a6-4631-891a-0ff5e92ed4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:01.792485 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896026f-e6a6-4631-891a-0ff5e92ed4c2 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:01.792758 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896026f-e6a6-4631-891a-0ff5e92ed4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:01.793023 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896026f-e6a6-4631-891a-0ff5e92ed4c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:01.804157 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f896026f-e6a6-4631-891a-0ff5e92ed4c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:15:01.809336 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-f896026f-e6a6-4631-891a-0ff5e92ed4c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:15:01.809336 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:01.809336 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:01.809336 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:01.809336 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:01.809336 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:01.809336 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:01.809336 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:01.809336 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:01.809336 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:01.809336 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:01.809336 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:01.809336 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:15:01.809336 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:15:01.809336 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:15:01.809336 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:15:01.810311 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:15:01.810311 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:15:01.810311 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:15:01.810311 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:15:01.810311 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:01.810311 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:01.810311 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:01.810311 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:01.810311 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:01.810311 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:01.810311 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:01.810311 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:01.810311 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:01.810311 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:01.810311 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:01.810311 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:01.811167 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:01.811167 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:01.811167 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:01.811167 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:01.811167 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:15:01.811167 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:15:01.811167 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:15:01.811167 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:15:01.811167 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:15:01.811167 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:15:01.811167 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:15:01.811167 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:01.811167 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 285/579] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Sep 9 20:15:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:01.811872 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f896026f-e6a6-4631-891a-0ff5e92ed4c2 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:01.844443 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a96486-a3bf-4ff2-b462-e597633d2bc7 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:01.882302 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb721e8-c041-490b-9c3c-2000da7f2531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:01.883017 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb721e8-c041-490b-9c3c-2000da7f2531 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:01.883302 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb721e8-c041-490b-9c3c-2000da7f2531 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:01.884269 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb721e8-c041-490b-9c3c-2000da7f2531 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:01.890823 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb721e8-c041-490b-9c3c-2000da7f2531 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:01.892395 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb721e8-c041-490b-9c3c-2000da7f2531 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:01.895079 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:01.903905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-ffb721e8-c041-490b-9c3c-2000da7f2531 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:01.903905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:01.903905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:01.903905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:01.903905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:01.903905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:01.903905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:01.903905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:01.903905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:01.903905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:01.903905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:01.903905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:01.903905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:15:01.903905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:15:01.903905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:15:01.904895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:15:01.904895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:01.904895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:01.904895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:01.904895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:01.904895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:01.904895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:01.904895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:01.904895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:01.904895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:01.904895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:01.904895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:01.904895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:01.904895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:01.904895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:01.904895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:01.904895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:01.905940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:15:01.905940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:15:01.905940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:15:01.905940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:15:01.905940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:15:01.905940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:15:01.905940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:01.905940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:01.905940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 295/580] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:15:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:01.906431 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb721e8-c041-490b-9c3c-2000da7f2531 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:01.920999 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f896026f-e6a6-4631-891a-0ff5e92ed4c2 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:01.949159 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:01.960642 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f813d-db31-4b4e-b892-318c729829ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:01.961515 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f813d-db31-4b4e-b892-318c729829ef None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:01.961788 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f813d-db31-4b4e-b892-318c729829ef None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:01.962054 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f813d-db31-4b4e-b892-318c729829ef None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:01.962714 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f813d-db31-4b4e-b892-318c729829ef None admin] RBAC: Adding query filter params (name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:01.963009 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f813d-db31-4b4e-b892-318c729829ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:01.964361 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f813d-db31-4b4e-b892-318c729829ef None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:01.976756 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 286/581] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Sep 9 20:15:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:01.976756 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f813d-db31-4b4e-b892-318c729829ef None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:02.011267 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb721e8-c041-490b-9c3c-2000da7f2531 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:02.049326 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a16e79-041e-4796-b121-f7bcb75b33a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:02.049956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a16e79-041e-4796-b121-f7bcb75b33a6 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:02.050231 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a16e79-041e-4796-b121-f7bcb75b33a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:02.050500 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a16e79-041e-4796-b121-f7bcb75b33a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:02.063512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a16e79-041e-4796-b121-f7bcb75b33a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:02.064947 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a16e79-041e-4796-b121-f7bcb75b33a6 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:02.080083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-59a16e79-041e-4796-b121-f7bcb75b33a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:02.080083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:02.080083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:02.080083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:02.080083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:02.080083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:02.080083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:02.080083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:02.080083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:02.080083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:02.080083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:02.080083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:02.080083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:02.080083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:02.080083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:02.080083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:02.086416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:02.086416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:02.086416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:02.086416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:02.086416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:02.086416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:02.086416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:02.086416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:02.086416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:02.086416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:02.086416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:02.086416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:02.086416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:02.086416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:02.086416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:02.086416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:02.086416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:02.087543 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:02.087543 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:02.087543 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:02.087543 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 296/582] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:02.087543 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a16e79-041e-4796-b121-f7bcb75b33a6 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:02.090017 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f813d-db31-4b4e-b892-318c729829ef None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:02.127612 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3b6c2-418a-4ad8-9928-c4af172bd843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:02.128271 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3b6c2-418a-4ad8-9928-c4af172bd843 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:02.128500 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3b6c2-418a-4ad8-9928-c4af172bd843 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:02.129333 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3b6c2-418a-4ad8-9928-c4af172bd843 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:02.129695 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3b6c2-418a-4ad8-9928-c4af172bd843 None admin] RBAC: Adding query filter params (name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:02.129992 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3b6c2-418a-4ad8-9928-c4af172bd843 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:02.131338 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3b6c2-418a-4ad8-9928-c4af172bd843 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:02.139708 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 287/583] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Sep 9 20:15:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:02.140363 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3b6c2-418a-4ad8-9928-c4af172bd843 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:02.144921 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:02.194577 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:02.203340 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a16e79-041e-4796-b121-f7bcb75b33a6 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:02.240612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f515f264-3d18-4df6-80a0-6cac0fb26c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:02.242023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515f264-3d18-4df6-80a0-6cac0fb26c34 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:02.242308 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515f264-3d18-4df6-80a0-6cac0fb26c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:02.242587 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515f264-3d18-4df6-80a0-6cac0fb26c34 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:02.243269 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f515f264-3d18-4df6-80a0-6cac0fb26c34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:02.243576 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f515f264-3d18-4df6-80a0-6cac0fb26c34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:02.244991 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f515f264-3d18-4df6-80a0-6cac0fb26c34 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:02.246385 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3b6c2-418a-4ad8-9928-c4af172bd843 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:02.261534 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 297/584] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:15:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:02.261534 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f515f264-3d18-4df6-80a0-6cac0fb26c34 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:02.286881 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a73e68-da38-4fe4-aae4-fde2e31a586c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:02.287509 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a73e68-da38-4fe4-aae4-fde2e31a586c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:02.287846 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a73e68-da38-4fe4-aae4-fde2e31a586c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:02.288100 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a73e68-da38-4fe4-aae4-fde2e31a586c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:02.298937 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a73e68-da38-4fe4-aae4-fde2e31a586c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:02.300407 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a73e68-da38-4fe4-aae4-fde2e31a586c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:02.316750 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-44a73e68-da38-4fe4-aae4-fde2e31a586c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:02.316750 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:02.316750 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:02.316750 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:02.316750 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:02.316750 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:02.316750 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:02.316750 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:02.316750 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:02.316750 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:02.316750 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:02.316750 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:02.316750 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:02.316750 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:02.316750 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:02.316750 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:02.317855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:02.317855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:02.317855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:02.317855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:02.317855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:02.317855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:02.317855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:02.317855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:02.317855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:02.317855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:02.317855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:02.317855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:02.317855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:02.317855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:02.317855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:02.317855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:02.317855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:02.318797 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:02.318797 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:02.318797 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:02.318797 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 288/585] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:02.319104 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a73e68-da38-4fe4-aae4-fde2e31a586c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:02.371749 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f515f264-3d18-4df6-80a0-6cac0fb26c34 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:02.377027 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:02.407926 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e027a-46cc-4e5a-b489-1a79bf4da944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:02.408556 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e027a-46cc-4e5a-b489-1a79bf4da944 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:02.408886 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e027a-46cc-4e5a-b489-1a79bf4da944 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:02.409222 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e027a-46cc-4e5a-b489-1a79bf4da944 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:02.409939 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e027a-46cc-4e5a-b489-1a79bf4da944 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:02.410301 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e027a-46cc-4e5a-b489-1a79bf4da944 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:02.411719 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e027a-46cc-4e5a-b489-1a79bf4da944 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:02.422379 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a73e68-da38-4fe4-aae4-fde2e31a586c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:02.427656 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 298/586] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:02.427656 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e027a-46cc-4e5a-b489-1a79bf4da944 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:02.440925 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:02.465861 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d854d061-c693-44c2-ad84-fa18d2d5e141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:02.466954 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854d061-c693-44c2-ad84-fa18d2d5e141 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:02.467238 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854d061-c693-44c2-ad84-fa18d2d5e141 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:02.467558 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854d061-c693-44c2-ad84-fa18d2d5e141 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:02.480322 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d854d061-c693-44c2-ad84-fa18d2d5e141 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:02.482014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d854d061-c693-44c2-ad84-fa18d2d5e141 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:02.497995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-d854d061-c693-44c2-ad84-fa18d2d5e141 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:02.497995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:02.497995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:02.497995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:02.497995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:02.497995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:02.497995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:02.497995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:02.497995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:02.497995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:02.497995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:02.497995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:02.497995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:02.497995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:02.497995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:02.497995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:02.499276 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:02.499276 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:02.499276 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:02.499276 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:02.499276 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:02.499276 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:02.499276 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:02.499276 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:02.499276 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:02.499276 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:02.499276 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:02.499276 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:02.499276 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:02.499276 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:02.499276 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:02.499276 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:02.499276 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:02.500502 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:02.500502 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:02.500502 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:02.503836 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 289/587] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:02.504856 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d854d061-c693-44c2-ad84-fa18d2d5e141 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:02.536975 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e027a-46cc-4e5a-b489-1a79bf4da944 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:02.573041 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facf8bc6-f610-4223-bb98-452e21556503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:02.573714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf8bc6-f610-4223-bb98-452e21556503 None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:02.573990 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf8bc6-f610-4223-bb98-452e21556503 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:02.574283 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf8bc6-f610-4223-bb98-452e21556503 None admin] PATH_INFO: `/v3/projects/933820e2adbc473bbd57a4d59be75d72/users/87ece6128f9245f7b49f3e7a9faaed22/roles/8ec32f702e0c447993ff294cbac536b1` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:02.592866 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facf8bc6-f610-4223-bb98-452e21556503 None admin] RBAC: Authorizing `identity:create_grant(project_id=933820e2adbc473bbd57a4d59be75d72, user_id=87ece6128f9245f7b49f3e7a9faaed22, role_id=8ec32f702e0c447993ff294cbac536b1)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:02.594326 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facf8bc6-f610-4223-bb98-452e21556503 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:02.607983 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 299/588] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Sep 9 20:15:02 2022] PUT /identity/v3/projects/933820e2adbc473bbd57a4d59be75d72/users/87ece6128f9245f7b49f3e7a9faaed22/roles/8ec32f702e0c447993ff294cbac536b1 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:15:02.608967 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facf8bc6-f610-4223-bb98-452e21556503 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:02.620714 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d854d061-c693-44c2-ad84-fa18d2d5e141 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:02.634075 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:02.684193 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e802c285-98f5-481a-8a6b-5ffde84bd515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:02.685177 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802c285-98f5-481a-8a6b-5ffde84bd515 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:02.685589 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802c285-98f5-481a-8a6b-5ffde84bd515 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:02.686103 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802c285-98f5-481a-8a6b-5ffde84bd515 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:02.687354 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e802c285-98f5-481a-8a6b-5ffde84bd515 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:02.687852 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e802c285-98f5-481a-8a6b-5ffde84bd515 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:02.689441 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e802c285-98f5-481a-8a6b-5ffde84bd515 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:02.691993 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:02.707061 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 290/589] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:02.708171 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e802c285-98f5-481a-8a6b-5ffde84bd515 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:02.738853 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facf8bc6-f610-4223-bb98-452e21556503 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:02.779099 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ccf8f-24be-43a3-9de5-f94787c0fe6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:02.779740 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ccf8f-24be-43a3-9de5-f94787c0fe6a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:02.780019 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ccf8f-24be-43a3-9de5-f94787c0fe6a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:02.780289 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ccf8f-24be-43a3-9de5-f94787c0fe6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:02.788777 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ccf8f-24be-43a3-9de5-f94787c0fe6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:02.790259 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ccf8f-24be-43a3-9de5-f94787c0fe6a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:02.804941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-770ccf8f-24be-43a3-9de5-f94787c0fe6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:02.804941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:02.804941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:02.804941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:02.804941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:02.804941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:02.804941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:02.804941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:02.804941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:02.804941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:02.804941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:02.804941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:02.804941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:15:02.804941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:15:02.804941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:15:02.806196 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:15:02.806196 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:02.806196 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:02.806196 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:02.806196 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:02.806196 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:02.806196 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:02.806196 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:02.806196 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:02.806196 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:02.806196 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:02.806196 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:02.806196 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:02.806196 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:02.806196 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:02.806196 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:02.806196 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:02.807315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:15:02.807315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:15:02.807315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:15:02.807315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:15:02.807315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:15:02.807315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:15:02.807315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:02.807315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:02.807315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 300/590] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:15:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:02.807315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ccf8f-24be-43a3-9de5-f94787c0fe6a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:02.840921 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e802c285-98f5-481a-8a6b-5ffde84bd515 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:02.888803 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815cb117-9202-434a-a087-4e984320a5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:02.889443 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815cb117-9202-434a-a087-4e984320a5e1 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:02.889722 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815cb117-9202-434a-a087-4e984320a5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:02.889988 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815cb117-9202-434a-a087-4e984320a5e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:02.890688 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815cb117-9202-434a-a087-4e984320a5e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:02.890984 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815cb117-9202-434a-a087-4e984320a5e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:02.892445 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815cb117-9202-434a-a087-4e984320a5e1 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:02.909332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 291/591] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:02.910410 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815cb117-9202-434a-a087-4e984320a5e1 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:02.933454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ccf8f-24be-43a3-9de5-f94787c0fe6a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:02.946970 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:02.962251 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:02.975688 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54f65d-8dfd-4388-8202-5019a0678c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:02.976685 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54f65d-8dfd-4388-8202-5019a0678c95 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:02.977422 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54f65d-8dfd-4388-8202-5019a0678c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:02.977867 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54f65d-8dfd-4388-8202-5019a0678c95 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:02.986956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a54f65d-8dfd-4388-8202-5019a0678c95 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:02.989232 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a54f65d-8dfd-4388-8202-5019a0678c95 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:03.002855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-9a54f65d-8dfd-4388-8202-5019a0678c95 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 20:15:03.002855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:03.002855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:03.002855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:03.002855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:03.002855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:03.002855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:03.002855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:03.002855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:03.002855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:03.002855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:03.002855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:03.002855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:15:03.002855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:15:03.002855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:15:03.002855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:15:03.004353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:03.004353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:03.004353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:15:03.004353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:03.004353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:15:03.004353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:03.004353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:03.004353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:03.004353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:03.004353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:03.004353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:03.004353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:03.004353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:03.004353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:03.004353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:03.004353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:03.004353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:03.004353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:03.005461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:03.005461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:03.005461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:15:03.005461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:15:03.005461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:15:03.005461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:15:03.005461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:15:03.005461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:15:03.005461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 20:15:03.005461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:03.007598 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 301/592] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Sep 9 20:15:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:03.008543 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54f65d-8dfd-4388-8202-5019a0678c95 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:03.043133 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815cb117-9202-434a-a087-4e984320a5e1 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:03.088026 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92987bc-195f-4340-b164-0bbfd2a9e524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:03.088901 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92987bc-195f-4340-b164-0bbfd2a9e524 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:03.089181 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92987bc-195f-4340-b164-0bbfd2a9e524 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:03.089479 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92987bc-195f-4340-b164-0bbfd2a9e524 None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:03.090171 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92987bc-195f-4340-b164-0bbfd2a9e524 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:03.090704 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92987bc-195f-4340-b164-0bbfd2a9e524 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:03.093042 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92987bc-195f-4340-b164-0bbfd2a9e524 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:03.114285 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 292/593] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:15:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:03.114285 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92987bc-195f-4340-b164-0bbfd2a9e524 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:03.129767 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54f65d-8dfd-4388-8202-5019a0678c95 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:03.173757 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6722a-8851-40da-91cc-d01b1e29d359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:03.175024 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6722a-8851-40da-91cc-d01b1e29d359 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:03.175510 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6722a-8851-40da-91cc-d01b1e29d359 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:03.176145 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6722a-8851-40da-91cc-d01b1e29d359 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:03.187790 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6722a-8851-40da-91cc-d01b1e29d359 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:03.190298 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6722a-8851-40da-91cc-d01b1e29d359 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:03.205706 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-39b6722a-8851-40da-91cc-d01b1e29d359 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 20:15:03.205706 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:03.205706 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:03.205706 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:03.205706 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:03.205706 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:03.205706 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:03.205706 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:03.205706 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:03.205706 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:03.205706 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:03.205706 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:03.205706 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:15:03.205706 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:15:03.205706 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:15:03.205706 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:15:03.206698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:03.206698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:03.206698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:15:03.206698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:03.206698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:15:03.206698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:03.206698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:03.206698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:03.206698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:03.206698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:03.206698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:03.206698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:03.206698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:03.206698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:03.206698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:03.206698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:03.206698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:03.206698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:03.207643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:03.207643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:03.207643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:15:03.207643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:15:03.207643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:15:03.207643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:15:03.207643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:15:03.207643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:15:03.207643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 20:15:03.207643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:03.210404 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 302/594] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:15:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:03.211377 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6722a-8851-40da-91cc-d01b1e29d359 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:03.228080 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:03.242406 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92987bc-195f-4340-b164-0bbfd2a9e524 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:03.279257 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a346147-3c71-490f-a99f-8fcffce8d045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:03.280038 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a346147-3c71-490f-a99f-8fcffce8d045 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:03.280417 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a346147-3c71-490f-a99f-8fcffce8d045 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:03.280790 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a346147-3c71-490f-a99f-8fcffce8d045 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:03.281642 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a346147-3c71-490f-a99f-8fcffce8d045 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:03.281997 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:03.282155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a346147-3c71-490f-a99f-8fcffce8d045 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:03.283855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a346147-3c71-490f-a99f-8fcffce8d045 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:03.309161 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 293/595] 10.222.0.26 () {56 vars in 1139 bytes} [Fri Sep 9 20:15:03 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:03.309161 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a346147-3c71-490f-a99f-8fcffce8d045 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:03.353770 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6722a-8851-40da-91cc-d01b1e29d359 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:03.389541 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f349c5-0bc3-4fd8-80a0-ff652369387e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:03.390356 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f349c5-0bc3-4fd8-80a0-ff652369387e None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:03.390736 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f349c5-0bc3-4fd8-80a0-ff652369387e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:03.391110 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f349c5-0bc3-4fd8-80a0-ff652369387e None admin] PATH_INFO: `/v3/projects/933820e2adbc473bbd57a4d59be75d72/users/11d86d2df1cb485592e8a5d7387131ee/roles/8ec32f702e0c447993ff294cbac536b1` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:03.407176 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f349c5-0bc3-4fd8-80a0-ff652369387e None admin] RBAC: Authorizing `identity:create_grant(project_id=933820e2adbc473bbd57a4d59be75d72, user_id=11d86d2df1cb485592e8a5d7387131ee, role_id=8ec32f702e0c447993ff294cbac536b1)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:03.408848 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f349c5-0bc3-4fd8-80a0-ff652369387e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:03.429480 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 303/596] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Sep 9 20:15:03 2022] PUT /identity/v3/projects/933820e2adbc473bbd57a4d59be75d72/users/11d86d2df1cb485592e8a5d7387131ee/roles/8ec32f702e0c447993ff294cbac536b1 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:15:03.429480 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f349c5-0bc3-4fd8-80a0-ff652369387e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:03.432778 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a346147-3c71-490f-a99f-8fcffce8d045 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:03.510706 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfb3136-07c2-4bec-a611-67e7a9e65545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:03.511676 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb3136-07c2-4bec-a611-67e7a9e65545 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:03.512154 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb3136-07c2-4bec-a611-67e7a9e65545 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:03.512592 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb3136-07c2-4bec-a611-67e7a9e65545 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:03.514238 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfb3136-07c2-4bec-a611-67e7a9e65545 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:03.514735 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfb3136-07c2-4bec-a611-67e7a9e65545 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:03.521562 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfb3136-07c2-4bec-a611-67e7a9e65545 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:03.566122 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 294/597] 10.222.0.26 () {56 vars in 1137 bytes} [Fri Sep 9 20:15:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:03.571118 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfb3136-07c2-4bec-a611-67e7a9e65545 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:03.589757 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:03.620729 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:03.629647 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f349c5-0bc3-4fd8-80a0-ff652369387e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:03.685320 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6416d942-956a-4b36-b2d2-cfcb3b6be7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:03.686348 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416d942-956a-4b36-b2d2-cfcb3b6be7a2 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:03.686793 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416d942-956a-4b36-b2d2-cfcb3b6be7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:03.687219 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416d942-956a-4b36-b2d2-cfcb3b6be7a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:03.699885 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6416d942-956a-4b36-b2d2-cfcb3b6be7a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:03.701376 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6416d942-956a-4b36-b2d2-cfcb3b6be7a2 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:03.717575 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-6416d942-956a-4b36-b2d2-cfcb3b6be7a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:03.717575 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:03.717575 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:03.717575 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:03.717575 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:03.717575 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:03.717575 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:03.717575 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:03.717575 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:03.717575 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:03.717575 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:03.717575 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:03.717575 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:03.717575 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:03.717575 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:03.717575 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:03.718759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:03.718759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:03.718759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:03.718759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:03.718759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:03.718759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:03.718759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:03.718759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:03.718759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:03.718759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:03.718759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:03.718759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:03.718759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:03.718759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:03.718759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:03.718759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:03.718759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:03.719911 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:03.719911 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:03.719911 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:03.719911 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 304/598] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:03.720197 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6cc13c-7798-444f-90ce-6a762ac0beb9 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:03.724734 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6cc13c-7798-444f-90ce-6a762ac0beb9 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:03.724734 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6cc13c-7798-444f-90ce-6a762ac0beb9 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:03.724734 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 305/599] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:03.725651 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6cc13c-7798-444f-90ce-6a762ac0beb9 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:03.743702 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfb3136-07c2-4bec-a611-67e7a9e65545 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:03.788900 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3936b-3f1f-4ff2-8762-34f142504700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:03.790060 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3936b-3f1f-4ff2-8762-34f142504700 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:03.790731 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3936b-3f1f-4ff2-8762-34f142504700 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:03.791154 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3936b-3f1f-4ff2-8762-34f142504700 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:03.806148 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb3936b-3f1f-4ff2-8762-34f142504700 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:03.807817 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb3936b-3f1f-4ff2-8762-34f142504700 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:03.828155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-7cb3936b-3f1f-4ff2-8762-34f142504700 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:03.828155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:03.828155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:03.828155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:03.828155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:03.828155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:03.828155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:03.828155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:03.828155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:03.828155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:03.828155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:03.828155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:03.828155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:03.828155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:03.828155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:03.828155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:03.829322 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:03.829322 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:03.829322 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:03.829322 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:03.829322 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:03.829322 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:03.829322 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:03.829322 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:03.829322 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:03.829322 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:03.829322 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:03.829322 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:03.829322 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:03.829322 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:03.829322 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:03.829322 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:03.829322 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:03.830302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:03.830302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:03.830302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:03.832415 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 295/600] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:03.834505 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f5c61-e1de-44b9-8681-5554082e8c81 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:03.835199 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f5c61-e1de-44b9-8681-5554082e8c81 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:03.835629 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f5c61-e1de-44b9-8681-5554082e8c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:03.840132 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6cc13c-7798-444f-90ce-6a762ac0beb9 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:03.859585 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-bf8f5c61-e1de-44b9-8681-5554082e8c81 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:03.861036 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-bf8f5c61-e1de-44b9-8681-5554082e8c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:03.883912 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de72fcfd-efab-4851-8a5c-d2f271e6062c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:03.884794 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de72fcfd-efab-4851-8a5c-d2f271e6062c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:03.885223 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de72fcfd-efab-4851-8a5c-d2f271e6062c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:03.885676 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de72fcfd-efab-4851-8a5c-d2f271e6062c None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:03.887036 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de72fcfd-efab-4851-8a5c-d2f271e6062c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:03.887510 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de72fcfd-efab-4851-8a5c-d2f271e6062c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:03.889338 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de72fcfd-efab-4851-8a5c-d2f271e6062c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:03.909257 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 306/601] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:03.910413 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de72fcfd-efab-4851-8a5c-d2f271e6062c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:03.920918 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:03.922556 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:03.949401 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 296/602] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:15:03.950575 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f5c61-e1de-44b9-8681-5554082e8c81 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:04.026289 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de72fcfd-efab-4851-8a5c-d2f271e6062c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:04.062804 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460c598-d834-48b0-b23e-f66169f4f122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:04.063338 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460c598-d834-48b0-b23e-f66169f4f122 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:04.063619 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460c598-d834-48b0-b23e-f66169f4f122 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:04.063891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460c598-d834-48b0-b23e-f66169f4f122 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:04.067481 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a460c598-d834-48b0-b23e-f66169f4f122 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:04.067481 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a460c598-d834-48b0-b23e-f66169f4f122 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:04.067481 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a460c598-d834-48b0-b23e-f66169f4f122 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:04.079760 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 307/603] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:04.090432 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460c598-d834-48b0-b23e-f66169f4f122 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:04.123982 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f5c61-e1de-44b9-8681-5554082e8c81 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:04.186868 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:04.210555 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571bd8b-ca21-4d88-aa21-118814dfd5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:04.211384 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571bd8b-ca21-4d88-aa21-118814dfd5f2 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:04.211514 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571bd8b-ca21-4d88-aa21-118814dfd5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:04.211738 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571bd8b-ca21-4d88-aa21-118814dfd5f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:04.227757 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460c598-d834-48b0-b23e-f66169f4f122 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:04.230288 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5571bd8b-ca21-4d88-aa21-118814dfd5f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:04.231783 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5571bd8b-ca21-4d88-aa21-118814dfd5f2 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:04.262883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-5571bd8b-ca21-4d88-aa21-118814dfd5f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:04.262883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:04.262883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:04.262883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:04.262883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:04.262883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:04.262883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:04.262883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:04.262883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:04.262883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:04.262883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:04.262883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:04.262883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:15:04.262883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:15:04.262883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:15:04.263859 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:15:04.263859 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:04.263859 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:04.263859 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:04.263859 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:04.263859 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:04.263859 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:04.263859 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:04.263859 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:04.263859 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:04.263859 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:04.263859 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:04.263859 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:04.263859 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:04.263859 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:04.263859 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:04.263859 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:04.264801 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:15:04.264801 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:15:04.264801 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:15:04.264801 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:15:04.264801 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:15:04.264801 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:15:04.264801 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:04.264801 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:04.264801 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 297/604] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:15:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:04.270375 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571bd8b-ca21-4d88-aa21-118814dfd5f2 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:04.277080 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e5fb59-93be-4972-aff5-14150e45e518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:04.277966 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5fb59-93be-4972-aff5-14150e45e518 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:04.278461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5fb59-93be-4972-aff5-14150e45e518 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:04.278880 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5fb59-93be-4972-aff5-14150e45e518 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:04.292462 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e5fb59-93be-4972-aff5-14150e45e518 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:04.294131 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e5fb59-93be-4972-aff5-14150e45e518 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:04.311067 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-d8e5fb59-93be-4972-aff5-14150e45e518 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:04.311067 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:04.311067 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:04.311067 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:04.311067 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:04.311067 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:04.311067 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:04.311067 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:04.311067 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:04.311067 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:04.311067 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:04.311067 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:04.311067 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:15:04.311067 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:15:04.311067 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:15:04.312209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:15:04.312209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:04.312209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:04.312209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:04.312209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:04.312209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:04.312209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:04.312209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:04.312209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:04.312209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:04.312209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:04.312209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:04.312209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:04.312209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:04.312209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:04.312209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:04.312209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:04.313331 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:15:04.313331 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:15:04.313331 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:15:04.313331 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:15:04.313331 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:15:04.313331 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:15:04.313331 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:04.313331 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:04.315241 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 308/605] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:15:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:04.316257 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e5fb59-93be-4972-aff5-14150e45e518 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:04.344154 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:04.414360 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571bd8b-ca21-4d88-aa21-118814dfd5f2 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:04.422312 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e5fb59-93be-4972-aff5-14150e45e518 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:04.448423 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:04.452027 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cdf0a2-21da-40a9-ae7c-f0347bc32b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:04.452836 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cdf0a2-21da-40a9-ae7c-f0347bc32b0e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:04.453285 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cdf0a2-21da-40a9-ae7c-f0347bc32b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:04.453746 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cdf0a2-21da-40a9-ae7c-f0347bc32b0e None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:04.455218 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cdf0a2-21da-40a9-ae7c-f0347bc32b0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:04.455957 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cdf0a2-21da-40a9-ae7c-f0347bc32b0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:04.457829 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cdf0a2-21da-40a9-ae7c-f0347bc32b0e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:04.458119 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc23e0f-0b7d-4f98-bda9-ec407762ec31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:04.458751 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc23e0f-0b7d-4f98-bda9-ec407762ec31 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:04.459030 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc23e0f-0b7d-4f98-bda9-ec407762ec31 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:04.459302 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc23e0f-0b7d-4f98-bda9-ec407762ec31 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:04.470431 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc23e0f-0b7d-4f98-bda9-ec407762ec31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:15:04.472442 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 298/606] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:15:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:04.473428 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cdf0a2-21da-40a9-ae7c-f0347bc32b0e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:04.479792 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-8bc23e0f-0b7d-4f98-bda9-ec407762ec31 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:15:04.479792 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:04.479792 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:04.479792 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:04.479792 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:04.479792 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:04.479792 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:04.479792 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:04.479792 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:04.479792 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:04.479792 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:04.479792 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:04.479792 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:15:04.479792 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:15:04.479792 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:15:04.479792 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:15:04.480803 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:15:04.480803 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:15:04.480803 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:15:04.480803 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:15:04.480803 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:04.480803 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:04.480803 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:04.480803 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:04.480803 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:04.480803 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:04.480803 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:04.480803 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:04.480803 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:04.480803 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:04.480803 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:04.480803 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:04.481718 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:04.481718 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:04.481718 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:04.481718 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:04.481718 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:15:04.481718 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:15:04.481718 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:15:04.481718 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:15:04.481718 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:15:04.481718 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:15:04.481718 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:15:04.481718 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:04.481718 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 309/607] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Sep 9 20:15:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:04.481718 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc23e0f-0b7d-4f98-bda9-ec407762ec31 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:04.603125 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cdf0a2-21da-40a9-ae7c-f0347bc32b0e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:04.610967 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:04.626152 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc23e0f-0b7d-4f98-bda9-ec407762ec31 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:04.644368 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46506150-2a95-4264-b58f-6119ac4cac76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:04.645673 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46506150-2a95-4264-b58f-6119ac4cac76 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:04.646151 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46506150-2a95-4264-b58f-6119ac4cac76 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:04.646971 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46506150-2a95-4264-b58f-6119ac4cac76 None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:04.648007 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46506150-2a95-4264-b58f-6119ac4cac76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:04.648506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46506150-2a95-4264-b58f-6119ac4cac76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:04.650417 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46506150-2a95-4264-b58f-6119ac4cac76 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:04.671564 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 299/608] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:15:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:04.672567 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46506150-2a95-4264-b58f-6119ac4cac76 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:04.680134 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1beec9c-918f-4188-8515-f3fbac437b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:04.681066 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1beec9c-918f-4188-8515-f3fbac437b6d None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:04.681555 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1beec9c-918f-4188-8515-f3fbac437b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:04.682003 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1beec9c-918f-4188-8515-f3fbac437b6d None admin] PATH_INFO: `/v3/projects/933820e2adbc473bbd57a4d59be75d72/users/ee5c6e51f2694bf5ae7ea097cf33fb3b/roles/8ec32f702e0c447993ff294cbac536b1` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:04.702898 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1beec9c-918f-4188-8515-f3fbac437b6d None admin] RBAC: Authorizing `identity:create_grant(project_id=933820e2adbc473bbd57a4d59be75d72, user_id=ee5c6e51f2694bf5ae7ea097cf33fb3b, role_id=8ec32f702e0c447993ff294cbac536b1)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:04.705866 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1beec9c-918f-4188-8515-f3fbac437b6d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:04.729332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 310/609] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Sep 9 20:15:04 2022] PUT /identity/v3/projects/933820e2adbc473bbd57a4d59be75d72/users/ee5c6e51f2694bf5ae7ea097cf33fb3b/roles/8ec32f702e0c447993ff294cbac536b1 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:15:04.730828 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1beec9c-918f-4188-8515-f3fbac437b6d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:04.813573 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:04.856204 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46506150-2a95-4264-b58f-6119ac4cac76 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:04.896284 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1beec9c-918f-4188-8515-f3fbac437b6d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:04.905851 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e9152-f84f-4fbb-bc67-7d344200069f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:04.906481 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e9152-f84f-4fbb-bc67-7d344200069f None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:04.906760 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e9152-f84f-4fbb-bc67-7d344200069f None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:04.907611 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e9152-f84f-4fbb-bc67-7d344200069f None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:04.908322 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e9152-f84f-4fbb-bc67-7d344200069f None admin] RBAC: Adding query filter params (name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:04.909067 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e9152-f84f-4fbb-bc67-7d344200069f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:04.911235 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e9152-f84f-4fbb-bc67-7d344200069f None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:04.922397 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 300/610] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Sep 9 20:15:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:04.923829 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc1b12-b872-4942-b9c0-d28ca2f3d6e4 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:04.924192 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc1b12-b872-4942-b9c0-d28ca2f3d6e4 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:04.924555 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc1b12-b872-4942-b9c0-d28ca2f3d6e4 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:04.926126 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 301/611] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:04.927760 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc1b12-b872-4942-b9c0-d28ca2f3d6e4 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:04.941225 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad64eee-5fd4-48c5-a289-8c1d1daa138b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:04.941961 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad64eee-5fd4-48c5-a289-8c1d1daa138b None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:04.942239 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad64eee-5fd4-48c5-a289-8c1d1daa138b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:04.942557 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad64eee-5fd4-48c5-a289-8c1d1daa138b None admin] PATH_INFO: `/v3/projects/933820e2adbc473bbd57a4d59be75d72/users/5298486310004b63b51116dbc747e31d/roles/8ec32f702e0c447993ff294cbac536b1` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:04.958763 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad64eee-5fd4-48c5-a289-8c1d1daa138b None admin] RBAC: Authorizing `identity:create_grant(project_id=933820e2adbc473bbd57a4d59be75d72, user_id=5298486310004b63b51116dbc747e31d, role_id=8ec32f702e0c447993ff294cbac536b1)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:04.960169 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:04.960746 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad64eee-5fd4-48c5-a289-8c1d1daa138b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:04.979637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 311/612] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Sep 9 20:15:04 2022] PUT /identity/v3/projects/933820e2adbc473bbd57a4d59be75d72/users/5298486310004b63b51116dbc747e31d/roles/8ec32f702e0c447993ff294cbac536b1 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:15:04.980935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031b99d-ea3f-4e08-8161-50afa8efc0d2 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:04.981224 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031b99d-ea3f-4e08-8161-50afa8efc0d2 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:04.981678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031b99d-ea3f-4e08-8161-50afa8efc0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:05.006482 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-6031b99d-ea3f-4e08-8161-50afa8efc0d2 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:05.008134 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6031b99d-ea3f-4e08-8161-50afa8efc0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:05.048560 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc1b12-b872-4942-b9c0-d28ca2f3d6e4 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:05.085640 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e1b9ea-9bc6-416d-a19f-485bcfc72130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:05.086277 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1b9ea-9bc6-416d-a19f-485bcfc72130 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:05.086555 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1b9ea-9bc6-416d-a19f-485bcfc72130 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:05.086823 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1b9ea-9bc6-416d-a19f-485bcfc72130 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:05.096338 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 312/613] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:15:05.105157 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e1b9ea-9bc6-416d-a19f-485bcfc72130 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:05.106894 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e1b9ea-9bc6-416d-a19f-485bcfc72130 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:05.131155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-82e1b9ea-9bc6-416d-a19f-485bcfc72130 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:05.131155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:05.131155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:05.131155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:05.131155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:05.131155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:05.131155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:05.131155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:05.131155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:05.131155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:05.131155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:05.131155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:05.131155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:05.131155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:05.131155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:05.131155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:05.132334 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:05.132334 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:05.132334 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:05.132334 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:05.132334 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:05.132334 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:05.132334 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:05.132334 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:05.132334 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:05.132334 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:05.132334 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:05.132334 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:05.132334 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:05.132334 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:05.132334 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:05.132334 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:05.132334 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:05.133324 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:05.133324 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:05.133324 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:05.133324 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 302/614] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:05.138232 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6031b99d-ea3f-4e08-8161-50afa8efc0d2 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:05.185828 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:05.278157 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6031b99d-ea3f-4e08-8161-50afa8efc0d2 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:05.313519 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f9be5-388b-472d-86db-b72cff842edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:05.314288 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f9be5-388b-472d-86db-b72cff842edc None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:05.314665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f9be5-388b-472d-86db-b72cff842edc None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:05.315038 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f9be5-388b-472d-86db-b72cff842edc None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:05.315785 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f9be5-388b-472d-86db-b72cff842edc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:05.316188 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f9be5-388b-472d-86db-b72cff842edc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:05.317674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f9be5-388b-472d-86db-b72cff842edc None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:05.330483 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 313/615] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:05.335962 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e1b9ea-9bc6-416d-a19f-485bcfc72130 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:05.406424 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f9be5-388b-472d-86db-b72cff842edc None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:05.489426 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:05.508663 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:05.516503 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f9be5-388b-472d-86db-b72cff842edc None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:05.552741 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107dc503-8c1d-4c8a-ab14-74ef9b639427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:05.553686 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107dc503-8c1d-4c8a-ab14-74ef9b639427 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:05.554246 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107dc503-8c1d-4c8a-ab14-74ef9b639427 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:05.554656 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107dc503-8c1d-4c8a-ab14-74ef9b639427 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:05.563170 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e1b9ea-9bc6-416d-a19f-485bcfc72130 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:05.565308 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107dc503-8c1d-4c8a-ab14-74ef9b639427 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:15:05.571441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-107dc503-8c1d-4c8a-ab14-74ef9b639427 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:15:05.571441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:05.571441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:05.571441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:05.571441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:05.571441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:05.571441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:05.571441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:05.571441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:05.571441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:05.571441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:05.571441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:05.571441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:15:05.571441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:15:05.571441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:15:05.571441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:15:05.572564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:15:05.572564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:15:05.572564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:15:05.572564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:15:05.572564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:05.572564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:05.572564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:05.572564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:05.572564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:05.572564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:05.572564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:05.572564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:05.572564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:05.572564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:05.572564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:05.572564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:05.573612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:05.573612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:05.573612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:05.573612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:05.573612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:15:05.573612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:15:05.573612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:15:05.573612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:15:05.573612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:15:05.573612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:15:05.573612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:15:05.573612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:05.575907 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 314/616] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Sep 9 20:15:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:05.581635 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107dc503-8c1d-4c8a-ab14-74ef9b639427 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:05.612866 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d7cce-7178-4676-adc5-e3938280e477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:05.613777 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d7cce-7178-4676-adc5-e3938280e477 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:05.614164 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d7cce-7178-4676-adc5-e3938280e477 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:05.614557 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d7cce-7178-4676-adc5-e3938280e477 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:05.623428 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664d7cce-7178-4676-adc5-e3938280e477 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:05.625026 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664d7cce-7178-4676-adc5-e3938280e477 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:05.638655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-664d7cce-7178-4676-adc5-e3938280e477 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 20:15:05.638655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:05.638655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:05.638655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:05.638655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:05.638655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:05.638655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:05.638655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:05.638655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:05.638655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:05.638655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:05.638655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:05.638655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:15:05.638655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:15:05.638655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:15:05.638655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:15:05.639848 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:05.639848 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:05.639848 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:15:05.639848 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:05.639848 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:15:05.639848 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:05.639848 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:05.639848 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:05.639848 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:05.639848 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:05.639848 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:05.639848 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:05.639848 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:05.639848 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:05.639848 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:05.639848 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:05.639848 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:05.639848 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:05.641022 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:05.641022 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:05.641022 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:15:05.641022 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:15:05.641022 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:15:05.641022 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:15:05.641022 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:15:05.641022 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:15:05.641022 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 20:15:05.641022 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:05.642884 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 303/617] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:15:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:05.648478 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d7cce-7178-4676-adc5-e3938280e477 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:05.696182 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107dc503-8c1d-4c8a-ab14-74ef9b639427 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:05.757977 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7508b74-d47c-485b-a037-408b1a903958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:05.758595 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7508b74-d47c-485b-a037-408b1a903958 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:05.758872 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7508b74-d47c-485b-a037-408b1a903958 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:05.759141 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7508b74-d47c-485b-a037-408b1a903958 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:05.759808 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7508b74-d47c-485b-a037-408b1a903958 None admin] RBAC: Adding query filter params (name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:05.760105 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7508b74-d47c-485b-a037-408b1a903958 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:05.761488 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7508b74-d47c-485b-a037-408b1a903958 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:05.764970 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:05.787603 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 315/618] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Sep 9 20:15:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:05.794214 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7508b74-d47c-485b-a037-408b1a903958 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:05.807307 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d7cce-7178-4676-adc5-e3938280e477 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:05.844096 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9260c89-4dad-47de-a2d6-df3ce452fc3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:05.845111 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9260c89-4dad-47de-a2d6-df3ce452fc3d None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:05.845651 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9260c89-4dad-47de-a2d6-df3ce452fc3d None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:05.846133 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9260c89-4dad-47de-a2d6-df3ce452fc3d None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:05.847127 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9260c89-4dad-47de-a2d6-df3ce452fc3d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:05.847665 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9260c89-4dad-47de-a2d6-df3ce452fc3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:05.849378 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9260c89-4dad-47de-a2d6-df3ce452fc3d None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:05.873270 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:05.873828 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 304/619] 10.222.0.26 () {56 vars in 1135 bytes} [Fri Sep 9 20:15:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:05.875905 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9260c89-4dad-47de-a2d6-df3ce452fc3d None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:05.990116 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9260c89-4dad-47de-a2d6-df3ce452fc3d None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:05.993219 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7508b74-d47c-485b-a037-408b1a903958 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:06.028415 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b26659-7262-4ebf-b060-f25240866166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:06.029050 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b26659-7262-4ebf-b060-f25240866166 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:06.029355 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b26659-7262-4ebf-b060-f25240866166 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:06.029629 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b26659-7262-4ebf-b060-f25240866166 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:06.030894 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c8961c-0747-43f9-890a-badc764441c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:06.031939 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8961c-0747-43f9-890a-badc764441c7 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:06.032545 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8961c-0747-43f9-890a-badc764441c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:06.033030 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8961c-0747-43f9-890a-badc764441c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:06.040859 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b26659-7262-4ebf-b060-f25240866166 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:06.042327 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b26659-7262-4ebf-b060-f25240866166 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:06.045309 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c8961c-0747-43f9-890a-badc764441c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:06.046960 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c8961c-0747-43f9-890a-badc764441c7 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:06.057545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-f7b26659-7262-4ebf-b060-f25240866166 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:06.057545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:06.057545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:06.057545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:06.057545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:06.057545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:06.057545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:06.057545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:06.057545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:06.057545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:06.057545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:06.057545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:06.057545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:06.057545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:06.057545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:06.057545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:06.058538 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:06.058538 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:06.058538 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:06.058538 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:06.058538 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:06.058538 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:06.058538 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:06.058538 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:06.058538 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:06.058538 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:06.058538 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:06.058538 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:06.058538 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:06.058538 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:06.058538 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:06.058538 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:06.058538 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:06.059449 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:06.059449 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:06.059449 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:06.059449 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 316/620] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:06.064142 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b26659-7262-4ebf-b060-f25240866166 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:06.066785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-85c8961c-0747-43f9-890a-badc764441c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:06.066785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:06.066785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:06.066785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:06.066785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:06.066785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:06.066785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:06.066785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:06.066785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:06.066785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:06.066785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:06.066785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:06.066785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:06.066785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:06.066785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:06.066785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:06.067881 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:06.067881 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:06.067881 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:06.067881 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:06.067881 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:06.067881 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:06.067881 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:06.067881 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:06.067881 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:06.067881 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:06.067881 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:06.067881 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:06.067881 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:06.067881 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:06.067881 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:06.067881 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:06.067881 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:06.069003 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:06.069003 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:06.069003 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:06.075364 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 305/621] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:06.075364 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b71c2d-444c-4386-a9ed-8a41a48c0276 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:06.075364 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b71c2d-444c-4386-a9ed-8a41a48c0276 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:06.075364 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b71c2d-444c-4386-a9ed-8a41a48c0276 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:06.078226 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 306/622] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:06.079110 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b71c2d-444c-4386-a9ed-8a41a48c0276 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:06.125645 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:06.141832 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:06.180890 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b26659-7262-4ebf-b060-f25240866166 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:06.205578 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b71c2d-444c-4386-a9ed-8a41a48c0276 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:06.233054 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686f959e-de83-42bb-87b5-26179e144886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:06.233855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f959e-de83-42bb-87b5-26179e144886 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:06.234239 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f959e-de83-42bb-87b5-26179e144886 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:06.234618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f959e-de83-42bb-87b5-26179e144886 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:06.235371 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686f959e-de83-42bb-87b5-26179e144886 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:06.235781 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686f959e-de83-42bb-87b5-26179e144886 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:06.237228 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686f959e-de83-42bb-87b5-26179e144886 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:06.260788 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 317/623] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:06.260788 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d3e43-c5f9-4b36-a3ed-a9defae3cc25 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:06.260788 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d3e43-c5f9-4b36-a3ed-a9defae3cc25 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:06.260788 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d3e43-c5f9-4b36-a3ed-a9defae3cc25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:06.261832 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae8bb3e-8da6-4eef-b1ac-f37f64451ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:06.262555 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8bb3e-8da6-4eef-b1ac-f37f64451ec5 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:06.262956 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8bb3e-8da6-4eef-b1ac-f37f64451ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:06.263313 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8bb3e-8da6-4eef-b1ac-f37f64451ec5 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:06.264069 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae8bb3e-8da6-4eef-b1ac-f37f64451ec5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:06.264477 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae8bb3e-8da6-4eef-b1ac-f37f64451ec5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:06.266038 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae8bb3e-8da6-4eef-b1ac-f37f64451ec5 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:06.281092 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-296d3e43-c5f9-4b36-a3ed-a9defae3cc25 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:06.282650 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-296d3e43-c5f9-4b36-a3ed-a9defae3cc25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:06.285022 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 307/624] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:06.286189 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae8bb3e-8da6-4eef-b1ac-f37f64451ec5 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:06.369625 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 318/625] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:15:06.370725 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d3e43-c5f9-4b36-a3ed-a9defae3cc25 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:06.418103 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae8bb3e-8da6-4eef-b1ac-f37f64451ec5 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:06.434381 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:06.441704 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:06.492576 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ded540-a755-4e97-a1d6-53c2e6936d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:06.493210 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ded540-a755-4e97-a1d6-53c2e6936d65 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:06.493508 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ded540-a755-4e97-a1d6-53c2e6936d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:06.493778 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ded540-a755-4e97-a1d6-53c2e6936d65 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:06.506592 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ded540-a755-4e97-a1d6-53c2e6936d65 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:06.508400 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ded540-a755-4e97-a1d6-53c2e6936d65 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:06.526083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d3e43-c5f9-4b36-a3ed-a9defae3cc25 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:06.529978 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-e3ded540-a755-4e97-a1d6-53c2e6936d65 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:15:06.529978 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:06.529978 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:06.529978 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:06.529978 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:06.529978 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:06.529978 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:06.529978 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:06.529978 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:06.529978 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:06.529978 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:06.529978 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:06.529978 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:15:06.529978 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:15:06.529978 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:15:06.529978 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:15:06.531143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:06.531143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:06.531143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:15:06.531143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:06.531143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:15:06.531143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:06.531143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:06.531143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:06.531143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:06.531143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:06.531143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:06.531143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:06.531143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:06.531143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:06.531143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:06.531143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:06.531143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:06.531143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:06.532135 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:06.532135 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:06.532135 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:15:06.532135 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:15:06.532135 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:15:06.532135 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:15:06.532135 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:15:06.532135 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:15:06.532135 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:15:06.532135 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:06.541942 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 308/626] 10.222.0.26 () {56 vars in 1093 bytes} [Fri Sep 9 20:15:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:06.548257 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ded540-a755-4e97-a1d6-53c2e6936d65 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:06.567027 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23970bdf-5dbe-41c1-95a8-d65d661a2335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:06.567668 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23970bdf-5dbe-41c1-95a8-d65d661a2335 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:06.567944 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23970bdf-5dbe-41c1-95a8-d65d661a2335 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:06.568213 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23970bdf-5dbe-41c1-95a8-d65d661a2335 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:06.575173 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23970bdf-5dbe-41c1-95a8-d65d661a2335 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:06.576603 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23970bdf-5dbe-41c1-95a8-d65d661a2335 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:06.589949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-23970bdf-5dbe-41c1-95a8-d65d661a2335 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:06.589949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:06.589949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:06.589949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:06.589949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:06.589949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:06.589949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:06.589949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:06.589949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:06.589949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:06.589949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:06.589949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:06.589949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:15:06.589949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:15:06.589949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:15:06.590905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:15:06.590905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:06.590905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:06.590905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:06.590905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:06.590905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:06.590905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:06.590905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:06.590905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:06.590905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:06.590905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:06.590905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:06.590905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:06.590905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:06.590905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:06.590905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:06.590905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:06.591821 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:15:06.591821 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:15:06.591821 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:15:06.591821 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:15:06.591821 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:15:06.591821 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:15:06.591821 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:06.591821 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:06.591821 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 319/627] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:15:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:06.597284 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23970bdf-5dbe-41c1-95a8-d65d661a2335 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:06.703714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:06.706708 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23970bdf-5dbe-41c1-95a8-d65d661a2335 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:06.736905 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ded540-a755-4e97-a1d6-53c2e6936d65 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:06.751354 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60733c7f-6e5d-4af1-a4ef-73c6d01455f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:06.752074 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60733c7f-6e5d-4af1-a4ef-73c6d01455f5 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:06.752265 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60733c7f-6e5d-4af1-a4ef-73c6d01455f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:06.752507 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60733c7f-6e5d-4af1-a4ef-73c6d01455f5 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:06.753281 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60733c7f-6e5d-4af1-a4ef-73c6d01455f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:06.753628 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60733c7f-6e5d-4af1-a4ef-73c6d01455f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:06.755294 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60733c7f-6e5d-4af1-a4ef-73c6d01455f5 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:06.768145 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 320/628] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:15:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:06.773422 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60733c7f-6e5d-4af1-a4ef-73c6d01455f5 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:06.786808 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96586a11-20a3-4ea9-ac5c-902d024696fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:06.787622 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96586a11-20a3-4ea9-ac5c-902d024696fc None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:06.788057 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96586a11-20a3-4ea9-ac5c-902d024696fc None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:06.788431 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96586a11-20a3-4ea9-ac5c-902d024696fc None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:06.789289 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96586a11-20a3-4ea9-ac5c-902d024696fc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:06.789704 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96586a11-20a3-4ea9-ac5c-902d024696fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:06.791293 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96586a11-20a3-4ea9-ac5c-902d024696fc None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:06.814945 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 309/629] 10.222.0.26 () {56 vars in 1133 bytes} [Fri Sep 9 20:15:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:06.819739 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96586a11-20a3-4ea9-ac5c-902d024696fc None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:06.856556 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:06.900054 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60733c7f-6e5d-4af1-a4ef-73c6d01455f5 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:06.939823 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96586a11-20a3-4ea9-ac5c-902d024696fc None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:06.947703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e8938-17ae-4ea1-8b79-246d7f07a59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:06.949087 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e8938-17ae-4ea1-8b79-246d7f07a59b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:06.949605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e8938-17ae-4ea1-8b79-246d7f07a59b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:06.950078 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e8938-17ae-4ea1-8b79-246d7f07a59b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:06.964407 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e8938-17ae-4ea1-8b79-246d7f07a59b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:15:06.972461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-9f0e8938-17ae-4ea1-8b79-246d7f07a59b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:15:06.972461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:06.972461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:06.972461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:06.972461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:06.972461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:06.972461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:06.972461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:06.972461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:06.972461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:06.972461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:06.972461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:06.972461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:15:06.972461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:15:06.972461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:15:06.972461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:15:06.973422 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:15:06.973422 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:15:06.973422 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:15:06.973422 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:15:06.973422 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:06.973422 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:06.973422 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:06.973422 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:06.973422 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:06.973422 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:06.973422 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:06.973422 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:06.973422 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:06.973422 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:06.973422 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:06.973422 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:06.974283 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:06.974283 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:06.974283 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:06.974283 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:06.974283 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:15:06.974283 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:15:06.974283 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:15:06.974283 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:15:06.974283 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:15:06.974283 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:15:06.974283 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:15:06.974283 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:06.975469 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 321/630] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Sep 9 20:15:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:06.976926 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e8938-17ae-4ea1-8b79-246d7f07a59b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:06.981059 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f40da8-87c4-44f7-b53b-cae9f0c8218c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:06.981391 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f40da8-87c4-44f7-b53b-cae9f0c8218c None admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:06.981670 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f40da8-87c4-44f7-b53b-cae9f0c8218c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:06.981944 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f40da8-87c4-44f7-b53b-cae9f0c8218c None admin] PATH_INFO: `/v3/projects/933820e2adbc473bbd57a4d59be75d72/users/fcb62d06f7e94e9295c54504ddabb1d4/roles/8ec32f702e0c447993ff294cbac536b1` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:06.997061 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f40da8-87c4-44f7-b53b-cae9f0c8218c None admin] RBAC: Authorizing `identity:create_grant(project_id=933820e2adbc473bbd57a4d59be75d72, user_id=fcb62d06f7e94e9295c54504ddabb1d4, role_id=8ec32f702e0c447993ff294cbac536b1)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:06.998548 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f40da8-87c4-44f7-b53b-cae9f0c8218c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:07.004708 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:07.016300 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 310/631] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Sep 9 20:15:06 2022] PUT /identity/v3/projects/933820e2adbc473bbd57a4d59be75d72/users/fcb62d06f7e94e9295c54504ddabb1d4/roles/8ec32f702e0c447993ff294cbac536b1 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:15:07.017540 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f40da8-87c4-44f7-b53b-cae9f0c8218c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:07.152146 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e8938-17ae-4ea1-8b79-246d7f07a59b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:07.190043 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6f279a-ade1-4f75-ab7a-6ceba35ef1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:07.190660 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f279a-ade1-4f75-ab7a-6ceba35ef1a4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:07.190937 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f279a-ade1-4f75-ab7a-6ceba35ef1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:07.191208 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f279a-ade1-4f75-ab7a-6ceba35ef1a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:07.201306 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:07.229012 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f40da8-87c4-44f7-b53b-cae9f0c8218c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:07.274208 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1021d1e-6656-4e2f-90c6-420bd4a6c949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:07.274819 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1021d1e-6656-4e2f-90c6-420bd4a6c949 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:07.275093 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1021d1e-6656-4e2f-90c6-420bd4a6c949 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:07.275359 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1021d1e-6656-4e2f-90c6-420bd4a6c949 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:07.276016 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1021d1e-6656-4e2f-90c6-420bd4a6c949 None admin] RBAC: Adding query filter params (name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:07.276313 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1021d1e-6656-4e2f-90c6-420bd4a6c949 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:07.277701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1021d1e-6656-4e2f-90c6-420bd4a6c949 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:07.278212 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6f279a-ade1-4f75-ab7a-6ceba35ef1a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:07.280585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6f279a-ade1-4f75-ab7a-6ceba35ef1a4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:07.286410 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 311/632] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Sep 9 20:15:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:07.287671 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5db24-df1f-45d0-b1da-d2d7a24a4398 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:07.287953 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5db24-df1f-45d0-b1da-d2d7a24a4398 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:07.288227 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5db24-df1f-45d0-b1da-d2d7a24a4398 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:07.297280 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 312/633] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:07.297280 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f5db24-df1f-45d0-b1da-d2d7a24a4398 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:07.311892 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:07.407527 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 322/634] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:15:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:15:07.408717 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3120c1-a419-480f-865a-a1b7493264b7 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:07.408999 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3120c1-a419-480f-865a-a1b7493264b7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:07.409306 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3120c1-a419-480f-865a-a1b7493264b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:07.417925 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f5db24-df1f-45d0-b1da-d2d7a24a4398 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:07.430930 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-bd3120c1-a419-480f-865a-a1b7493264b7 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:07.432149 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-bd3120c1-a419-480f-865a-a1b7493264b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:07.465349 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa4b3f4-0920-47c0-8b87-86d257aeb497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:07.467242 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa4b3f4-0920-47c0-8b87-86d257aeb497 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:07.467700 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa4b3f4-0920-47c0-8b87-86d257aeb497 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:07.468142 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa4b3f4-0920-47c0-8b87-86d257aeb497 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:07.479802 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:07.483337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa4b3f4-0920-47c0-8b87-86d257aeb497 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:07.485949 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa4b3f4-0920-47c0-8b87-86d257aeb497 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:07.505021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-efa4b3f4-0920-47c0-8b87-86d257aeb497 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:07.505021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:07.505021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:07.505021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:07.505021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:07.505021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:07.505021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:07.505021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:07.505021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:07.505021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:07.505021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:07.505021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:07.505021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:07.505021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:07.505021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:07.505021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:07.506067 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:07.506067 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:07.506067 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:07.506067 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:07.506067 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:07.506067 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:07.506067 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:07.506067 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:07.506067 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:07.506067 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:07.506067 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:07.506067 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:07.506067 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:07.506067 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:07.506067 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:07.506067 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:07.506067 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:07.507059 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:07.507059 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:07.507059 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:07.507059 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 313/635] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:07.513453 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa4b3f4-0920-47c0-8b87-86d257aeb497 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:07.525276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 323/636] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:15:07.525276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3120c1-a419-480f-865a-a1b7493264b7 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:07.624583 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:07.688764 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3120c1-a419-480f-865a-a1b7493264b7 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:07.725962 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf77c65-9425-480e-8aa1-05145ed640a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:07.727567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf77c65-9425-480e-8aa1-05145ed640a4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:07.727989 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf77c65-9425-480e-8aa1-05145ed640a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:07.728462 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf77c65-9425-480e-8aa1-05145ed640a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:07.729307 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf77c65-9425-480e-8aa1-05145ed640a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:07.729748 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf77c65-9425-480e-8aa1-05145ed640a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:07.732481 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf77c65-9425-480e-8aa1-05145ed640a4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:07.745015 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 324/637] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:07.750733 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf77c65-9425-480e-8aa1-05145ed640a4 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:07.772105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa4b3f4-0920-47c0-8b87-86d257aeb497 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:07.809480 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2cbf49-d71c-4067-9aa3-d4ff69b4f10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:07.810185 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2cbf49-d71c-4067-9aa3-d4ff69b4f10f None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:07.810384 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2cbf49-d71c-4067-9aa3-d4ff69b4f10f None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:07.810652 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2cbf49-d71c-4067-9aa3-d4ff69b4f10f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:07.822401 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2cbf49-d71c-4067-9aa3-d4ff69b4f10f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:07.823864 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2cbf49-d71c-4067-9aa3-d4ff69b4f10f None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:07.838936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-8d2cbf49-d71c-4067-9aa3-d4ff69b4f10f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:07.838936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:07.838936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:07.838936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:07.838936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:07.838936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:07.838936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:07.838936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:07.838936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:07.838936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:07.838936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:07.838936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:07.838936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:07.838936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:07.838936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:07.838936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:07.839944 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:07.839944 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:07.839944 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:07.839944 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:07.839944 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:07.839944 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:07.839944 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:07.839944 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:07.839944 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:07.839944 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:07.839944 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:07.839944 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:07.839944 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:07.839944 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:07.839944 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:07.839944 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:07.839944 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:07.840864 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:07.840864 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:07.840864 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:07.840864 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 314/638] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:07.842822 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3a93a-c632-49d7-ae8e-2fb264a28762 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:07.842822 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3a93a-c632-49d7-ae8e-2fb264a28762 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:07.842822 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3a93a-c632-49d7-ae8e-2fb264a28762 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:07.843696 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 315/639] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:07.844499 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3a93a-c632-49d7-ae8e-2fb264a28762 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:07.869511 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf77c65-9425-480e-8aa1-05145ed640a4 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:07.880461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:07.896531 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:07.910329 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd47de-7e91-4c0c-bc71-10e2b3ca6d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:07.910963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd47de-7e91-4c0c-bc71-10e2b3ca6d05 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:07.911246 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd47de-7e91-4c0c-bc71-10e2b3ca6d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:07.911517 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd47de-7e91-4c0c-bc71-10e2b3ca6d05 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:07.918918 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd47de-7e91-4c0c-bc71-10e2b3ca6d05 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:07.920405 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd47de-7e91-4c0c-bc71-10e2b3ca6d05 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:07.933329 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-37dd47de-7e91-4c0c-bc71-10e2b3ca6d05 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 20:15:07.933329 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:07.933329 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:07.933329 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:07.933329 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:07.933329 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:07.933329 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:07.933329 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:07.933329 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:07.933329 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:07.933329 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:07.933329 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:07.933329 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:15:07.933329 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:15:07.933329 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:15:07.933329 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:15:07.934275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:07.934275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:07.934275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:15:07.934275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:07.934275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:15:07.934275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:07.934275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:07.934275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:07.934275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:07.934275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:07.934275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:07.934275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:07.934275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:07.934275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:07.934275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:07.934275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:07.934275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:07.934275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:07.935274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:07.935274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:07.935274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:15:07.935274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:15:07.935274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:15:07.935274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:15:07.935274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:15:07.935274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:15:07.935274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 20:15:07.935274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:07.939130 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 325/640] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:15:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:07.939833 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd47de-7e91-4c0c-bc71-10e2b3ca6d05 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:07.957425 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3a93a-c632-49d7-ae8e-2fb264a28762 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:07.994362 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30475e7-7f18-4f0b-8fa5-288e2effd7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:07.995388 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30475e7-7f18-4f0b-8fa5-288e2effd7b7 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:07.995894 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30475e7-7f18-4f0b-8fa5-288e2effd7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:07.996371 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30475e7-7f18-4f0b-8fa5-288e2effd7b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:08.009908 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30475e7-7f18-4f0b-8fa5-288e2effd7b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:15:08.015483 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-d30475e7-7f18-4f0b-8fa5-288e2effd7b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:15:08.015483 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:08.015483 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:08.015483 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:08.015483 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:08.015483 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:08.015483 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:08.015483 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:08.015483 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:08.015483 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:08.015483 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:08.015483 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:08.015483 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:15:08.015483 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:15:08.015483 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:15:08.015483 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:15:08.016495 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:15:08.016495 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:15:08.016495 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:15:08.016495 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:15:08.016495 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:08.016495 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:08.016495 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:08.016495 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:08.016495 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:08.016495 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:08.016495 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:08.016495 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:08.016495 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:08.016495 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:08.016495 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:08.016495 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:08.017412 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:08.017412 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:08.017412 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:08.017412 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:08.017412 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:15:08.017412 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:15:08.017412 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:15:08.017412 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:15:08.017412 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:15:08.017412 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:15:08.017412 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:15:08.017412 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:08.019349 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 316/641] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Sep 9 20:15:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:08.020674 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d611d9-85a0-4c66-af16-1ebf9412bf27 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:08.020953 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d611d9-85a0-4c66-af16-1ebf9412bf27 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:08.021229 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d611d9-85a0-4c66-af16-1ebf9412bf27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:08.040752 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-88d611d9-85a0-4c66-af16-1ebf9412bf27 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:08.041973 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-88d611d9-85a0-4c66-af16-1ebf9412bf27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:08.061803 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd47de-7e91-4c0c-bc71-10e2b3ca6d05 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:08.096464 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd84b5de-5119-429b-b945-72193451b905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:08.097075 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84b5de-5119-429b-b945-72193451b905 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:08.097411 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84b5de-5119-429b-b945-72193451b905 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:08.097687 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84b5de-5119-429b-b945-72193451b905 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:08.098356 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd84b5de-5119-429b-b945-72193451b905 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:08.098658 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd84b5de-5119-429b-b945-72193451b905 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:08.100557 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd84b5de-5119-429b-b945-72193451b905 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:08.110887 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 317/642] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:15:08.111477 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d611d9-85a0-4c66-af16-1ebf9412bf27 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:08.123200 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 326/643] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:08.123864 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd84b5de-5119-429b-b945-72193451b905 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:08.156762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:08.167528 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:08.242586 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd84b5de-5119-429b-b945-72193451b905 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:08.250180 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d611d9-85a0-4c66-af16-1ebf9412bf27 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:08.279473 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b518df2-e1a3-44cb-8aa0-8fa96b66ddad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:08.280033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b518df2-e1a3-44cb-8aa0-8fa96b66ddad None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:08.280313 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b518df2-e1a3-44cb-8aa0-8fa96b66ddad None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:08.280587 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b518df2-e1a3-44cb-8aa0-8fa96b66ddad None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:08.281263 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b518df2-e1a3-44cb-8aa0-8fa96b66ddad None admin] RBAC: Adding query filter params (name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:08.281575 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b518df2-e1a3-44cb-8aa0-8fa96b66ddad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:08.283192 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b518df2-e1a3-44cb-8aa0-8fa96b66ddad None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:08.292110 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 327/644] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Sep 9 20:15:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:08.292751 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b518df2-e1a3-44cb-8aa0-8fa96b66ddad None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:08.295617 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55059aea-bb8b-4ed8-b88d-1df05b266d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:08.297408 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55059aea-bb8b-4ed8-b88d-1df05b266d84 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:08.297889 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55059aea-bb8b-4ed8-b88d-1df05b266d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:08.298353 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55059aea-bb8b-4ed8-b88d-1df05b266d84 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:08.299271 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55059aea-bb8b-4ed8-b88d-1df05b266d84 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:08.299783 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55059aea-bb8b-4ed8-b88d-1df05b266d84 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:08.301409 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55059aea-bb8b-4ed8-b88d-1df05b266d84 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:08.325708 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 318/645] 10.222.0.26 () {56 vars in 1137 bytes} [Fri Sep 9 20:15:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:08.327469 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55059aea-bb8b-4ed8-b88d-1df05b266d84 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:08.394480 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:08.405742 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b518df2-e1a3-44cb-8aa0-8fa96b66ddad None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:08.442329 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:08.444263 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c791121a-3046-4676-a2c7-009ed3ef6b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:08.444975 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791121a-3046-4676-a2c7-009ed3ef6b76 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:08.445413 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791121a-3046-4676-a2c7-009ed3ef6b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:08.445780 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791121a-3046-4676-a2c7-009ed3ef6b76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:08.452198 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55059aea-bb8b-4ed8-b88d-1df05b266d84 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:08.452639 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c791121a-3046-4676-a2c7-009ed3ef6b76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:08.454208 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c791121a-3046-4676-a2c7-009ed3ef6b76 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:08.465563 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-c791121a-3046-4676-a2c7-009ed3ef6b76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:08.465563 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:08.465563 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:08.465563 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:08.465563 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:08.465563 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:08.465563 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:08.465563 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:08.465563 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:08.465563 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:08.465563 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:08.465563 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:08.465563 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:15:08.465563 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:15:08.465563 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:15:08.466703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:15:08.466703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:08.466703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:08.466703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:08.466703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:08.466703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:08.466703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:08.466703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:08.466703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:08.466703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:08.466703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:08.466703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:08.466703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:08.466703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:08.466703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:08.466703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:08.466703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:08.467982 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:15:08.467982 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:15:08.467982 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:15:08.467982 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:15:08.467982 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:15:08.467982 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:15:08.467982 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:08.467982 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:08.473288 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 328/646] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:15:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:08.473288 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c791121a-3046-4676-a2c7-009ed3ef6b76 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:08.495263 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c65060-7aca-41a4-acc7-c244667e5829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:08.495818 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c65060-7aca-41a4-acc7-c244667e5829 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:08.496097 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c65060-7aca-41a4-acc7-c244667e5829 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:08.496365 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c65060-7aca-41a4-acc7-c244667e5829 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:08.506421 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c65060-7aca-41a4-acc7-c244667e5829 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:08.507839 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c65060-7aca-41a4-acc7-c244667e5829 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:08.523109 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-38c65060-7aca-41a4-acc7-c244667e5829 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:08.523109 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:08.523109 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:08.523109 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:08.523109 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:08.523109 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:08.523109 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:08.523109 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:08.523109 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:08.523109 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:08.523109 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:08.523109 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:08.523109 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:08.523109 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:08.523109 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:08.523109 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:08.529039 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:08.529039 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:08.529039 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:08.529039 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:08.529039 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:08.529039 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:08.529039 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:08.529039 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:08.529039 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:08.529039 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:08.529039 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:08.529039 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:08.529039 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:08.529039 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:08.529039 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:08.529039 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:08.529039 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:08.530285 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:08.530285 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:08.530285 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:08.530285 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 319/647] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:08.530285 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c65060-7aca-41a4-acc7-c244667e5829 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:08.581312 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c791121a-3046-4676-a2c7-009ed3ef6b76 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:08.617741 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a032e4-8553-43ff-b6e0-f77b9af7892b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:08.618348 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a032e4-8553-43ff-b6e0-f77b9af7892b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:08.618625 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a032e4-8553-43ff-b6e0-f77b9af7892b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:08.618895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a032e4-8553-43ff-b6e0-f77b9af7892b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:08.630068 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c65060-7aca-41a4-acc7-c244667e5829 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:08.642395 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:08.669078 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57272e38-f0f3-49b1-820a-0e7a5266e5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:08.669634 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57272e38-f0f3-49b1-820a-0e7a5266e5e2 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:08.669916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57272e38-f0f3-49b1-820a-0e7a5266e5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:08.670190 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57272e38-f0f3-49b1-820a-0e7a5266e5e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:08.682683 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57272e38-f0f3-49b1-820a-0e7a5266e5e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:15:08.687893 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-57272e38-f0f3-49b1-820a-0e7a5266e5e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:15:08.687893 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:08.687893 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:08.687893 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:08.687893 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:08.687893 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:08.687893 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:08.687893 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:08.687893 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:08.687893 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:08.687893 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:08.687893 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:08.687893 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:15:08.687893 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:15:08.687893 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:15:08.687893 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:15:08.689065 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:15:08.689065 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:15:08.689065 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:15:08.689065 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:15:08.689065 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:08.689065 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:08.689065 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:08.689065 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:08.689065 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:08.689065 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:08.689065 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:08.689065 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:08.689065 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:08.689065 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:08.689065 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:08.689065 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:08.690332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:08.690332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:08.690332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:08.690332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:08.690332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:15:08.690332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:15:08.690332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:15:08.690332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:15:08.690332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:15:08.690332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:15:08.690332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:15:08.690332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:08.694532 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 320/648] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Sep 9 20:15:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:08.695374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57272e38-f0f3-49b1-820a-0e7a5266e5e2 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:08.700150 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:08.706753 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a032e4-8553-43ff-b6e0-f77b9af7892b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:08.708249 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a032e4-8553-43ff-b6e0-f77b9af7892b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:08.808384 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57272e38-f0f3-49b1-820a-0e7a5266e5e2 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:08.811454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 329/649] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:15:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:15:08.816908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a032e4-8553-43ff-b6e0-f77b9af7892b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:08.844552 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec6e7a-6692-4b64-b1a4-24a27eb18554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:08.845154 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec6e7a-6692-4b64-b1a4-24a27eb18554 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:08.845471 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec6e7a-6692-4b64-b1a4-24a27eb18554 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:08.845746 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec6e7a-6692-4b64-b1a4-24a27eb18554 None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:08.846436 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec6e7a-6692-4b64-b1a4-24a27eb18554 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:08.846742 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec6e7a-6692-4b64-b1a4-24a27eb18554 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:08.848173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec6e7a-6692-4b64-b1a4-24a27eb18554 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:08.865033 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 321/650] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:15:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:08.865033 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec6e7a-6692-4b64-b1a4-24a27eb18554 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:08.879226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:08.930111 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a032e4-8553-43ff-b6e0-f77b9af7892b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:08.932005 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:08.966967 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdafbf8-9979-4bec-9d97-adc6e5a7c13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:08.967728 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdafbf8-9979-4bec-9d97-adc6e5a7c13e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:08.968110 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdafbf8-9979-4bec-9d97-adc6e5a7c13e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:08.968482 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdafbf8-9979-4bec-9d97-adc6e5a7c13e None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:08.969454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdafbf8-9979-4bec-9d97-adc6e5a7c13e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:08.969852 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdafbf8-9979-4bec-9d97-adc6e5a7c13e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:08.971319 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdafbf8-9979-4bec-9d97-adc6e5a7c13e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:08.980761 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec6e7a-6692-4b64-b1a4-24a27eb18554 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:08.984462 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 330/651] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:08.985202 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdafbf8-9979-4bec-9d97-adc6e5a7c13e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:09.021348 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03c15b-9512-4135-9bc2-3fa4c6c456dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:09.022512 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03c15b-9512-4135-9bc2-3fa4c6c456dd None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:09.022798 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03c15b-9512-4135-9bc2-3fa4c6c456dd None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:09.023103 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03c15b-9512-4135-9bc2-3fa4c6c456dd None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:09.023739 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b03c15b-9512-4135-9bc2-3fa4c6c456dd None admin] RBAC: Adding query filter params (name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:09.024037 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b03c15b-9512-4135-9bc2-3fa4c6c456dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:09.025463 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b03c15b-9512-4135-9bc2-3fa4c6c456dd None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:09.034223 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 322/652] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Sep 9 20:15:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:09.039921 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03c15b-9512-4135-9bc2-3fa4c6c456dd None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:09.093482 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdafbf8-9979-4bec-9d97-adc6e5a7c13e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:09.106954 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:09.132310 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d6d2e7-5521-4d95-b98d-e3ea13530b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:09.132943 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6d2e7-5521-4d95-b98d-e3ea13530b22 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:09.133227 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6d2e7-5521-4d95-b98d-e3ea13530b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:09.133530 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6d2e7-5521-4d95-b98d-e3ea13530b22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:09.144792 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03c15b-9512-4135-9bc2-3fa4c6c456dd None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:09.144955 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d6d2e7-5521-4d95-b98d-e3ea13530b22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:09.146361 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d6d2e7-5521-4d95-b98d-e3ea13530b22 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:09.160618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-71d6d2e7-5521-4d95-b98d-e3ea13530b22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:09.160618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:09.160618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:09.160618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:09.160618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:09.160618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:09.160618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:09.160618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:09.160618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:09.160618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:09.160618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:09.160618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:09.160618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:09.160618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:09.160618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:09.160618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:09.161660 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:09.161660 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:09.161660 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:09.161660 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:09.161660 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:09.161660 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:09.161660 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:09.161660 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:09.161660 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:09.161660 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:09.161660 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:09.161660 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:09.161660 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:09.161660 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:09.161660 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:09.161660 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:09.161660 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:09.163195 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:09.163195 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:09.163195 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:09.163195 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 331/653] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:09.163195 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d6d2e7-5521-4d95-b98d-e3ea13530b22 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:09.173216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:09.183241 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7b70c-546a-4636-a53e-b76e4dc8cc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:09.183877 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7b70c-546a-4636-a53e-b76e4dc8cc64 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:09.184156 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7b70c-546a-4636-a53e-b76e4dc8cc64 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:09.184428 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7b70c-546a-4636-a53e-b76e4dc8cc64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:09.185182 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf7b70c-546a-4636-a53e-b76e4dc8cc64 None admin] RBAC: Adding query filter params (role.id=8ec32f702e0c447993ff294cbac536b1, scope.project.id=933820e2adbc473bbd57a4d59be75d72, user.id=87ece6128f9245f7b49f3e7a9faaed22) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:09.185530 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf7b70c-546a-4636-a53e-b76e4dc8cc64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:09.187008 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf7b70c-546a-4636-a53e-b76e4dc8cc64 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:09.193548 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 323/654] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:15:09 2022] GET /identity/v3/role_assignments?user.id=87ece6128f9245f7b49f3e7a9faaed22&scope.project.id=933820e2adbc473bbd57a4d59be75d72&role.id=8ec32f702e0c447993ff294cbac536b1 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:09.194950 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7b70c-546a-4636-a53e-b76e4dc8cc64 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:09.307483 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d6d2e7-5521-4d95-b98d-e3ea13530b22 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:09.330632 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7b70c-546a-4636-a53e-b76e4dc8cc64 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:09.355048 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f24b9-f6c2-441a-b595-8006f3afeae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:09.355714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f24b9-f6c2-441a-b595-8006f3afeae8 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:09.355997 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f24b9-f6c2-441a-b595-8006f3afeae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:09.356274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f24b9-f6c2-441a-b595-8006f3afeae8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:09.365042 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f24b9-f6c2-441a-b595-8006f3afeae8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:09.366133 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b614500-5470-4875-8a03-df13ca48df46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:09.366614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f24b9-f6c2-441a-b595-8006f3afeae8 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:09.366779 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b614500-5470-4875-8a03-df13ca48df46 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:09.367057 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b614500-5470-4875-8a03-df13ca48df46 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:09.367330 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b614500-5470-4875-8a03-df13ca48df46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:09.379385 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b614500-5470-4875-8a03-df13ca48df46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:09.380871 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b614500-5470-4875-8a03-df13ca48df46 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:09.382340 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-1a8f24b9-f6c2-441a-b595-8006f3afeae8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 20:15:09.382340 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:09.382340 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:09.382340 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:09.382340 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:09.382340 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:09.382340 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:09.382340 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:09.382340 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:09.382340 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:09.382340 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:09.382340 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:09.382340 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:15:09.382340 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:15:09.382340 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:15:09.382340 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:15:09.383520 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:09.383520 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:09.383520 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:15:09.383520 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:09.383520 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:15:09.383520 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:09.383520 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:09.383520 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:09.383520 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:09.383520 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:09.383520 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:09.383520 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:09.383520 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:09.383520 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:09.383520 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:09.383520 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:09.383520 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:09.383520 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:09.384721 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:09.384721 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:09.384721 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:15:09.384721 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:15:09.384721 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:15:09.384721 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:15:09.384721 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:15:09.384721 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:15:09.384721 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 20:15:09.384721 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:09.391420 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 332/655] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Sep 9 20:15:09 2022] GET /identity/v3/users/neutron => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:09.391420 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f24b9-f6c2-441a-b595-8006f3afeae8 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:09.400849 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-2b614500-5470-4875-8a03-df13ca48df46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:09.400849 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:09.400849 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:09.400849 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:09.400849 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:09.400849 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:09.400849 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:09.400849 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:09.400849 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:09.400849 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:09.400849 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:09.400849 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:09.400849 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:09.400849 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:09.400849 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:09.400849 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:09.402331 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:09.402331 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:09.402331 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:09.402331 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:09.402331 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:09.402331 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:09.402331 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:09.402331 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:09.402331 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:09.402331 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:09.402331 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:09.402331 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:09.402331 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:09.402331 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:09.402331 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:09.402331 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:09.402331 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:09.403284 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:09.403284 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:09.403284 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:09.403284 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 324/656] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:09.403525 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c49f3c-6f5a-40f1-ab02-908106262a8a None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:09.403596 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c49f3c-6f5a-40f1-ab02-908106262a8a None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:09.404789 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c49f3c-6f5a-40f1-ab02-908106262a8a None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:09.406369 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 325/657] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:09.407135 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c49f3c-6f5a-40f1-ab02-908106262a8a None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:09.417549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:09.465598 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:09.509177 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f24b9-f6c2-441a-b595-8006f3afeae8 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:09.527555 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c49f3c-6f5a-40f1-ab02-908106262a8a None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:09.545701 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7586f93-129c-4e7d-a4e9-d70a5e38e28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:09.546312 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7586f93-129c-4e7d-a4e9-d70a5e38e28a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:09.546590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7586f93-129c-4e7d-a4e9-d70a5e38e28a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:09.546855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7586f93-129c-4e7d-a4e9-d70a5e38e28a None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:09.547489 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7586f93-129c-4e7d-a4e9-d70a5e38e28a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:09.547784 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7586f93-129c-4e7d-a4e9-d70a5e38e28a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:09.549119 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7586f93-129c-4e7d-a4e9-d70a5e38e28a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:09.561536 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 333/658] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:09.562349 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7586f93-129c-4e7d-a4e9-d70a5e38e28a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:09.568012 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b29f0-de10-4122-8de5-9825b143d15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:09.568814 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b29f0-de10-4122-8de5-9825b143d15f None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:09.569193 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b29f0-de10-4122-8de5-9825b143d15f None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:09.569590 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b29f0-de10-4122-8de5-9825b143d15f None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:09.570423 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507b29f0-de10-4122-8de5-9825b143d15f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:09.571251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507b29f0-de10-4122-8de5-9825b143d15f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:09.573133 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507b29f0-de10-4122-8de5-9825b143d15f None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:09.596805 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 326/659] 10.222.0.26 () {56 vars in 1139 bytes} [Fri Sep 9 20:15:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:09.596805 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a885c8-bbe6-4ff6-9d42-a2a1fb7f54e1 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:09.596805 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a885c8-bbe6-4ff6-9d42-a2a1fb7f54e1 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:09.596805 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a885c8-bbe6-4ff6-9d42-a2a1fb7f54e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:09.613803 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-96a885c8-bbe6-4ff6-9d42-a2a1fb7f54e1 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:09.614917 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-96a885c8-bbe6-4ff6-9d42-a2a1fb7f54e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:09.658619 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:09.683743 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7586f93-129c-4e7d-a4e9-d70a5e38e28a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:09.688509 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 327/660] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:15:09.689385 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a885c8-bbe6-4ff6-9d42-a2a1fb7f54e1 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:09.723870 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0761d7-c9e4-47d3-94a7-0faa5b0e0931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:09.724617 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0761d7-c9e4-47d3-94a7-0faa5b0e0931 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:09.725074 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0761d7-c9e4-47d3-94a7-0faa5b0e0931 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:09.725449 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0761d7-c9e4-47d3-94a7-0faa5b0e0931 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:09.726166 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0761d7-c9e4-47d3-94a7-0faa5b0e0931 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:09.726539 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0761d7-c9e4-47d3-94a7-0faa5b0e0931 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:09.727965 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0761d7-c9e4-47d3-94a7-0faa5b0e0931 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:09.734331 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:09.745490 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 334/661] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:09.746511 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0761d7-c9e4-47d3-94a7-0faa5b0e0931 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:09.827517 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a885c8-bbe6-4ff6-9d42-a2a1fb7f54e1 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:09.866827 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0761d7-c9e4-47d3-94a7-0faa5b0e0931 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:09.879676 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67c7f3-b756-4ef1-9607-57ebad8adbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:09.880628 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67c7f3-b756-4ef1-9607-57ebad8adbc5 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:09.881593 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67c7f3-b756-4ef1-9607-57ebad8adbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:09.882295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67c7f3-b756-4ef1-9607-57ebad8adbc5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:09.891643 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd67c7f3-b756-4ef1-9607-57ebad8adbc5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:09.893935 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd67c7f3-b756-4ef1-9607-57ebad8adbc5 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:09.904509 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e3f30-ab6a-47a6-8f03-89735d66a3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:09.905201 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e3f30-ab6a-47a6-8f03-89735d66a3a7 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:09.905567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e3f30-ab6a-47a6-8f03-89735d66a3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:09.905900 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e3f30-ab6a-47a6-8f03-89735d66a3a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:09.908173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-bd67c7f3-b756-4ef1-9607-57ebad8adbc5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:09.908173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:09.908173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:09.908173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:09.908173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:09.908173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:09.908173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:09.908173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:09.908173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:09.908173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:09.908173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:09.908173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:09.908173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:15:09.908173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:15:09.908173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:15:09.909258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:15:09.909258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:09.909258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:09.909258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:09.909258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:09.909258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:09.909258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:09.909258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:09.909258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:09.909258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:09.909258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:09.909258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:09.909258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:09.909258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:09.909258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:09.909258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:09.909258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:09.910326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:15:09.910326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:15:09.910326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:15:09.910326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:15:09.910326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:15:09.910326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:15:09.910326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:09.910326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:09.911006 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 328/662] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:15:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:09.911615 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67c7f3-b756-4ef1-9607-57ebad8adbc5 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:09.929230 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:09.993032 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e3f30-ab6a-47a6-8f03-89735d66a3a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:09.994521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e3f30-ab6a-47a6-8f03-89735d66a3a7 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:10.025672 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:10.036979 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67c7f3-b756-4ef1-9607-57ebad8adbc5 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:10.073629 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c54c48-9593-40c5-81f3-43cbf9a5fdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:10.074254 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c54c48-9593-40c5-81f3-43cbf9a5fdda None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:10.074529 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c54c48-9593-40c5-81f3-43cbf9a5fdda None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:10.074797 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c54c48-9593-40c5-81f3-43cbf9a5fdda None admin] PATH_INFO: `/v3/users/glance` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:10.081851 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c54c48-9593-40c5-81f3-43cbf9a5fdda None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:10.083352 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c54c48-9593-40c5-81f3-43cbf9a5fdda None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:10.095543 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-15c54c48-9593-40c5-81f3-43cbf9a5fdda None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 20:15:10.095543 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:10.095543 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:10.095543 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:10.095543 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:10.095543 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:10.095543 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:10.095543 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:10.095543 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:10.095543 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:10.095543 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:10.095543 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:10.095543 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:15:10.095543 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:15:10.095543 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:15:10.095543 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:15:10.096570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:10.096570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:10.096570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:15:10.096570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:10.096570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:15:10.096570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:10.096570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:10.096570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:10.096570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:10.096570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:10.096570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:10.096570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:10.096570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:10.096570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:10.096570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:10.096570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:10.096570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:10.096570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:10.097560 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:10.097560 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:10.097560 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:15:10.097560 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:15:10.097560 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:15:10.097560 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:15:10.097560 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:15:10.097560 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:15:10.097560 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 20:15:10.097560 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:10.097560 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 329/663] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:15:09 2022] GET /identity/v3/users/glance => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:10.097560 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c54c48-9593-40c5-81f3-43cbf9a5fdda None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:10.108380 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 335/664] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:15:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:15:10.109324 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e3f30-ab6a-47a6-8f03-89735d66a3a7 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:10.157694 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:10.213190 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c54c48-9593-40c5-81f3-43cbf9a5fdda None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:10.221749 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e3f30-ab6a-47a6-8f03-89735d66a3a7 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:10.247394 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34e050-cd8b-4110-bc98-3a7c00caa3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:10.248081 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34e050-cd8b-4110-bc98-3a7c00caa3ab None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:10.248417 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34e050-cd8b-4110-bc98-3a7c00caa3ab None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:10.248751 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34e050-cd8b-4110-bc98-3a7c00caa3ab None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:10.249530 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af34e050-cd8b-4110-bc98-3a7c00caa3ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:10.249891 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af34e050-cd8b-4110-bc98-3a7c00caa3ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:10.251414 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af34e050-cd8b-4110-bc98-3a7c00caa3ab None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:10.258187 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a23335-71eb-471f-b395-631a2849ebc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:10.259089 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a23335-71eb-471f-b395-631a2849ebc2 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:10.259217 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a23335-71eb-471f-b395-631a2849ebc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:10.259424 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a23335-71eb-471f-b395-631a2849ebc2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:10.268785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 330/665] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:15:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:10.268785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34e050-cd8b-4110-bc98-3a7c00caa3ab None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:10.276554 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a23335-71eb-471f-b395-631a2849ebc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:15:10.277119 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:10.282941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-33a23335-71eb-471f-b395-631a2849ebc2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:15:10.282941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:10.282941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:10.282941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:10.282941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:10.282941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:10.282941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:10.282941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:10.282941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:10.282941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:10.282941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:10.282941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:10.282941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:15:10.282941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:15:10.282941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:15:10.282941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:15:10.284056 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:15:10.284056 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:15:10.284056 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:15:10.284056 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:15:10.284056 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:10.284056 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:10.284056 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:10.284056 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:10.284056 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:10.284056 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:10.284056 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:10.284056 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:10.284056 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:10.284056 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:10.284056 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:10.284056 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:10.285940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:10.285940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:10.285940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:10.285940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:10.285940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:15:10.285940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:15:10.285940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:15:10.285940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:15:10.285940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:15:10.285940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:15:10.285940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:15:10.285940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:10.285940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 336/666] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Sep 9 20:15:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:10.285940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a23335-71eb-471f-b395-631a2849ebc2 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:10.372916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34e050-cd8b-4110-bc98-3a7c00caa3ab None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:10.402674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a23335-71eb-471f-b395-631a2849ebc2 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:10.405649 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:10.410456 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4dc37-788a-44b7-9a4b-68ccfb8dc391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:10.411078 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4dc37-788a-44b7-9a4b-68ccfb8dc391 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:10.411448 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4dc37-788a-44b7-9a4b-68ccfb8dc391 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:10.411779 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4dc37-788a-44b7-9a4b-68ccfb8dc391 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:10.412703 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f4dc37-788a-44b7-9a4b-68ccfb8dc391 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:10.413036 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f4dc37-788a-44b7-9a4b-68ccfb8dc391 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:10.414487 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f4dc37-788a-44b7-9a4b-68ccfb8dc391 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:10.432287 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 331/667] 10.222.0.26 () {56 vars in 1137 bytes} [Fri Sep 9 20:15:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:10.433419 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4dc37-788a-44b7-9a4b-68ccfb8dc391 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:10.446832 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4660802-bd10-4a71-9a31-5658920a1e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:10.447657 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4660802-bd10-4a71-9a31-5658920a1e5a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:10.448076 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4660802-bd10-4a71-9a31-5658920a1e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:10.448509 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4660802-bd10-4a71-9a31-5658920a1e5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:10.459908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4660802-bd10-4a71-9a31-5658920a1e5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:10.461521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4660802-bd10-4a71-9a31-5658920a1e5a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:10.477949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-d4660802-bd10-4a71-9a31-5658920a1e5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:10.477949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:10.477949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:10.477949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:10.477949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:10.477949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:10.477949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:10.477949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:10.477949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:10.477949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:10.477949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:10.477949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:10.477949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:10.477949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:10.477949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:10.477949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:10.479033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:10.479033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:10.479033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:10.479033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:10.479033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:10.479033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:10.479033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:10.479033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:10.479033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:10.479033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:10.479033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:10.479033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:10.479033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:10.479033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:10.479033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:10.479033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:10.479033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:10.480114 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:10.480114 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:10.480114 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:10.481902 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 337/668] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:10.482681 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4660802-bd10-4a71-9a31-5658920a1e5a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:10.508111 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:10.544358 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4dc37-788a-44b7-9a4b-68ccfb8dc391 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:10.580308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c0baa2-1caf-4a75-a679-43afaeb76fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:10.580923 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c0baa2-1caf-4a75-a679-43afaeb76fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:10.581203 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c0baa2-1caf-4a75-a679-43afaeb76fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:10.581507 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c0baa2-1caf-4a75-a679-43afaeb76fc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:10.582232 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c0baa2-1caf-4a75-a679-43afaeb76fc3 None admin] RBAC: Adding query filter params (role.id=8ec32f702e0c447993ff294cbac536b1, scope.project.id=933820e2adbc473bbd57a4d59be75d72, user.id=11d86d2df1cb485592e8a5d7387131ee) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:10.582535 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c0baa2-1caf-4a75-a679-43afaeb76fc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:10.583960 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c0baa2-1caf-4a75-a679-43afaeb76fc3 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:10.601282 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 332/669] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:15:10 2022] GET /identity/v3/role_assignments?user.id=11d86d2df1cb485592e8a5d7387131ee&scope.project.id=933820e2adbc473bbd57a4d59be75d72&role.id=8ec32f702e0c447993ff294cbac536b1 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:10.601526 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4660802-bd10-4a71-9a31-5658920a1e5a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:10.601639 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c0baa2-1caf-4a75-a679-43afaeb76fc3 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:10.642371 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3c5e5-077d-4c39-8f3c-3496b8794b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:10.643241 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3c5e5-077d-4c39-8f3c-3496b8794b10 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:10.645096 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3c5e5-077d-4c39-8f3c-3496b8794b10 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:10.645096 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3c5e5-077d-4c39-8f3c-3496b8794b10 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:10.646913 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c3c5e5-077d-4c39-8f3c-3496b8794b10 None admin] RBAC: Adding query filter params (name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:10.647829 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c3c5e5-077d-4c39-8f3c-3496b8794b10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:10.650101 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c3c5e5-077d-4c39-8f3c-3496b8794b10 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:10.660086 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 338/670] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Sep 9 20:15:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:10.661148 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3c5e5-077d-4c39-8f3c-3496b8794b10 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:10.686607 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:10.733091 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c0baa2-1caf-4a75-a679-43afaeb76fc3 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:10.765989 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:10.771386 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df90bb-b851-4c87-b827-2b42b7f3c8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:10.772173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df90bb-b851-4c87-b827-2b42b7f3c8e2 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:10.772455 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df90bb-b851-4c87-b827-2b42b7f3c8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:10.772730 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df90bb-b851-4c87-b827-2b42b7f3c8e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:10.817544 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3c5e5-077d-4c39-8f3c-3496b8794b10 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:10.861659 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df90bb-b851-4c87-b827-2b42b7f3c8e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:10.863143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df90bb-b851-4c87-b827-2b42b7f3c8e2 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:10.864453 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ffc84-49b0-48d5-b125-67f291b7b564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:10.865515 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ffc84-49b0-48d5-b125-67f291b7b564 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:10.866064 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ffc84-49b0-48d5-b125-67f291b7b564 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:10.866531 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ffc84-49b0-48d5-b125-67f291b7b564 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:10.867374 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43ffc84-49b0-48d5-b125-67f291b7b564 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:10.868294 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43ffc84-49b0-48d5-b125-67f291b7b564 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:10.869861 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43ffc84-49b0-48d5-b125-67f291b7b564 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:10.887820 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 339/671] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:10.887820 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ffc84-49b0-48d5-b125-67f291b7b564 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:10.974862 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:10.993315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ffc84-49b0-48d5-b125-67f291b7b564 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:10.997078 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 333/672] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:15:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:15:10.998070 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df90bb-b851-4c87-b827-2b42b7f3c8e2 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:11.036008 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a79a6a-3e85-44fa-82e2-ffff86ce4cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:11.036977 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a79a6a-3e85-44fa-82e2-ffff86ce4cea None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:11.038226 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:11.040161 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a79a6a-3e85-44fa-82e2-ffff86ce4cea None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:11.040606 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a79a6a-3e85-44fa-82e2-ffff86ce4cea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:11.055109 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a79a6a-3e85-44fa-82e2-ffff86ce4cea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:11.056909 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a79a6a-3e85-44fa-82e2-ffff86ce4cea None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:11.073322 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-b5a79a6a-3e85-44fa-82e2-ffff86ce4cea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:11.073322 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:11.073322 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:11.073322 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:11.073322 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:11.073322 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:11.073322 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:11.073322 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:11.073322 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:11.073322 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:11.073322 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:11.073322 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:11.073322 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:11.073322 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:11.073322 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:11.073322 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:11.074644 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:11.074644 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:11.074644 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:11.074644 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:11.074644 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:11.074644 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:11.074644 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:11.074644 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:11.074644 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:11.074644 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:11.074644 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:11.074644 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:11.074644 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:11.074644 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:11.074644 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:11.074644 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:11.074644 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:11.075795 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:11.075795 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:11.075795 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:11.077557 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 340/673] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:11.078715 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a79a6a-3e85-44fa-82e2-ffff86ce4cea None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:11.125649 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df90bb-b851-4c87-b827-2b42b7f3c8e2 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:11.175398 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f2a49-40d7-4e77-9007-d3957397c1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:11.176224 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f2a49-40d7-4e77-9007-d3957397c1e3 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:11.177110 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f2a49-40d7-4e77-9007-d3957397c1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:11.177513 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f2a49-40d7-4e77-9007-d3957397c1e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:11.188186 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9f2a49-40d7-4e77-9007-d3957397c1e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:11.189658 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9f2a49-40d7-4e77-9007-d3957397c1e3 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:11.193934 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a79a6a-3e85-44fa-82e2-ffff86ce4cea None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:11.201175 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-4f9f2a49-40d7-4e77-9007-d3957397c1e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:11.201175 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:11.201175 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:11.201175 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:11.201175 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:11.201175 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:11.201175 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:11.201175 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:11.201175 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:11.201175 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:11.201175 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:11.201175 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:11.201175 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:15:11.201175 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:15:11.201175 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:15:11.202107 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:15:11.202107 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:11.202107 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:11.202107 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:11.202107 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:11.202107 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:11.202107 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:11.202107 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:11.202107 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:11.202107 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:11.202107 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:11.202107 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:11.202107 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:11.202107 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:11.202107 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:11.202107 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:11.202107 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:11.203021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:15:11.203021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:15:11.203021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:15:11.203021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:15:11.203021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:15:11.203021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:15:11.203021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:11.203021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:11.203021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 334/674] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:15:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:11.203524 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f2a49-40d7-4e77-9007-d3957397c1e3 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:11.238462 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:11.246632 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e7ad37-2691-4a35-a372-3388bfce9df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:11.247230 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7ad37-2691-4a35-a372-3388bfce9df6 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:11.247557 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7ad37-2691-4a35-a372-3388bfce9df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:11.247830 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7ad37-2691-4a35-a372-3388bfce9df6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:11.260654 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e7ad37-2691-4a35-a372-3388bfce9df6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:11.262128 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e7ad37-2691-4a35-a372-3388bfce9df6 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:11.279303 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-c4e7ad37-2691-4a35-a372-3388bfce9df6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:11.279303 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:11.279303 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:11.279303 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:11.279303 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:11.279303 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:11.279303 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:11.279303 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:11.279303 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:11.279303 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:11.279303 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:11.279303 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:11.279303 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:11.279303 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:11.279303 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:11.279303 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:11.280333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:11.280333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:11.280333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:11.280333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:11.280333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:11.280333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:11.280333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:11.280333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:11.280333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:11.280333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:11.280333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:11.280333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:11.280333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:11.280333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:11.280333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:11.280333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:11.280333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:11.281276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:11.281276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:11.281276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:11.285839 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 341/675] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:11.286976 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e7ad37-2691-4a35-a372-3388bfce9df6 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:11.328067 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:11.338907 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f2a49-40d7-4e77-9007-d3957397c1e3 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:11.377438 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4391b2bd-830c-4aa1-9b88-1dd4027bf4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:11.378430 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391b2bd-830c-4aa1-9b88-1dd4027bf4bf None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:11.379171 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391b2bd-830c-4aa1-9b88-1dd4027bf4bf None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:11.379663 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391b2bd-830c-4aa1-9b88-1dd4027bf4bf None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:11.380606 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4391b2bd-830c-4aa1-9b88-1dd4027bf4bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:11.381813 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4391b2bd-830c-4aa1-9b88-1dd4027bf4bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:11.383427 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4391b2bd-830c-4aa1-9b88-1dd4027bf4bf None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:11.398173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 335/676] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:11.399374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4391b2bd-830c-4aa1-9b88-1dd4027bf4bf None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:11.404329 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e7ad37-2691-4a35-a372-3388bfce9df6 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:11.440189 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a959aae-5485-4c2e-b4eb-c3194695c404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:11.440881 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a959aae-5485-4c2e-b4eb-c3194695c404 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:11.441168 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a959aae-5485-4c2e-b4eb-c3194695c404 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:11.441548 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a959aae-5485-4c2e-b4eb-c3194695c404 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:11.442301 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a959aae-5485-4c2e-b4eb-c3194695c404 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:11.442651 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a959aae-5485-4c2e-b4eb-c3194695c404 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:11.444642 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a959aae-5485-4c2e-b4eb-c3194695c404 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:11.462323 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 342/677] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:15:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:11.462323 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a959aae-5485-4c2e-b4eb-c3194695c404 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:11.504558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:11.516155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4391b2bd-830c-4aa1-9b88-1dd4027bf4bf None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:11.554543 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfdae49-cb9a-4323-bf88-dca387e6f72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:11.555170 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfdae49-cb9a-4323-bf88-dca387e6f72b None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:11.555478 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfdae49-cb9a-4323-bf88-dca387e6f72b None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:11.555684 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfdae49-cb9a-4323-bf88-dca387e6f72b None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:11.556322 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfdae49-cb9a-4323-bf88-dca387e6f72b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:11.556625 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfdae49-cb9a-4323-bf88-dca387e6f72b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:11.558029 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfdae49-cb9a-4323-bf88-dca387e6f72b None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:11.565357 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a959aae-5485-4c2e-b4eb-c3194695c404 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:11.570079 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 336/678] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:11.570869 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfdae49-cb9a-4323-bf88-dca387e6f72b None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:11.588895 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:11.603523 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7474b-2e24-4fbd-94c3-d891e093f378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:11.604157 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7474b-2e24-4fbd-94c3-d891e093f378 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:11.604487 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7474b-2e24-4fbd-94c3-d891e093f378 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:11.604771 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7474b-2e24-4fbd-94c3-d891e093f378 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:11.612589 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b7474b-2e24-4fbd-94c3-d891e093f378 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:11.614101 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b7474b-2e24-4fbd-94c3-d891e093f378 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:11.628258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-18b7474b-2e24-4fbd-94c3-d891e093f378 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 20:15:11.628258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:11.628258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:11.628258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:11.628258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:11.628258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:11.628258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:11.628258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:11.628258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:11.628258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:11.628258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:11.628258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:11.628258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:15:11.628258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:15:11.628258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:15:11.628258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:15:11.629624 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:11.629624 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:11.629624 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:15:11.629624 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:11.629624 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:15:11.629624 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:11.629624 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:11.629624 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:11.629624 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:11.629624 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:11.629624 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:11.629624 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:11.629624 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:11.629624 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:11.629624 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:11.629624 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:11.629624 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:11.629624 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:11.630567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:11.630567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:11.630567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:15:11.630567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:15:11.630567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:15:11.630567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:15:11.630567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:15:11.630567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:15:11.630567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 20:15:11.630567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:11.634504 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 343/679] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:15:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:11.636005 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e8f58-d939-4db0-8587-b3265c0ce21e None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:11.636113 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e8f58-d939-4db0-8587-b3265c0ce21e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:11.636328 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e8f58-d939-4db0-8587-b3265c0ce21e None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:11.637738 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 344/680] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:11.642312 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e8f58-d939-4db0-8587-b3265c0ce21e None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:11.692183 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfdae49-cb9a-4323-bf88-dca387e6f72b None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:11.727346 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573676e7-34d5-44cd-80b0-762617beb23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:11.727983 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573676e7-34d5-44cd-80b0-762617beb23e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:11.728265 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573676e7-34d5-44cd-80b0-762617beb23e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:11.728546 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573676e7-34d5-44cd-80b0-762617beb23e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:11.729358 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573676e7-34d5-44cd-80b0-762617beb23e None admin] RBAC: Adding query filter params (role.id=8ec32f702e0c447993ff294cbac536b1, scope.project.id=933820e2adbc473bbd57a4d59be75d72, user.id=ee5c6e51f2694bf5ae7ea097cf33fb3b) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:11.729681 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573676e7-34d5-44cd-80b0-762617beb23e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:11.731107 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573676e7-34d5-44cd-80b0-762617beb23e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:11.745268 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 337/681] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:15:11 2022] GET /identity/v3/role_assignments?user.id=ee5c6e51f2694bf5ae7ea097cf33fb3b&scope.project.id=933820e2adbc473bbd57a4d59be75d72&role.id=8ec32f702e0c447993ff294cbac536b1 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:11.745268 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573676e7-34d5-44cd-80b0-762617beb23e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:11.781927 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e8f58-d939-4db0-8587-b3265c0ce21e None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:11.783892 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:11.826711 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7513d2b6-9e66-4e50-8cb4-5a857afc39f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:11.827387 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513d2b6-9e66-4e50-8cb4-5a857afc39f7 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:11.827666 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513d2b6-9e66-4e50-8cb4-5a857afc39f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:11.827981 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513d2b6-9e66-4e50-8cb4-5a857afc39f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:11.836406 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7513d2b6-9e66-4e50-8cb4-5a857afc39f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:11.837925 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7513d2b6-9e66-4e50-8cb4-5a857afc39f7 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:11.847532 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:11.851838 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-7513d2b6-9e66-4e50-8cb4-5a857afc39f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:11.851838 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:11.851838 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:11.851838 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:11.851838 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:11.851838 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:11.851838 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:11.851838 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:11.851838 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:11.851838 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:11.851838 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:11.851838 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:11.851838 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:15:11.851838 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:15:11.851838 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:15:11.852858 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:15:11.852858 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:11.852858 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:11.852858 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:11.852858 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:11.852858 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:11.852858 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:11.852858 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:11.852858 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:11.852858 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:11.852858 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:11.852858 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:11.852858 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:11.852858 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:11.852858 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:11.852858 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:11.852858 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:11.853805 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:15:11.853805 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:15:11.853805 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:15:11.853805 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:15:11.853805 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:15:11.853805 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:15:11.853805 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:11.853805 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:11.857767 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 345/682] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:15:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:11.859159 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f658c0-e60a-427b-be40-4445abad0919 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:11.859448 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f658c0-e60a-427b-be40-4445abad0919 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:11.859728 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f658c0-e60a-427b-be40-4445abad0919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:11.876872 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573676e7-34d5-44cd-80b0-762617beb23e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:11.880011 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-60f658c0-e60a-427b-be40-4445abad0919 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:11.881659 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-60f658c0-e60a-427b-be40-4445abad0919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:11.917531 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb7668b-1277-4556-8cf1-8faa5037db98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:11.918359 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb7668b-1277-4556-8cf1-8faa5037db98 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:11.918647 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb7668b-1277-4556-8cf1-8faa5037db98 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:11.919073 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb7668b-1277-4556-8cf1-8faa5037db98 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:11.921289 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb7668b-1277-4556-8cf1-8faa5037db98 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:11.921289 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb7668b-1277-4556-8cf1-8faa5037db98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:11.922861 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb7668b-1277-4556-8cf1-8faa5037db98 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:11.950724 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 338/683] 10.222.0.26 () {56 vars in 1135 bytes} [Fri Sep 9 20:15:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:11.951906 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb7668b-1277-4556-8cf1-8faa5037db98 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:11.978475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 346/684] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:15:11.979164 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f658c0-e60a-427b-be40-4445abad0919 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:12.069793 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb7668b-1277-4556-8cf1-8faa5037db98 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:12.090996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:12.103501 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f658c0-e60a-427b-be40-4445abad0919 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:12.105128 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a602b-a45a-41c6-8f4d-c00da1ecb70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:12.105811 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a602b-a45a-41c6-8f4d-c00da1ecb70e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:12.106090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a602b-a45a-41c6-8f4d-c00da1ecb70e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:12.106373 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a602b-a45a-41c6-8f4d-c00da1ecb70e None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:12.107041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a602b-a45a-41c6-8f4d-c00da1ecb70e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:12.107342 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a602b-a45a-41c6-8f4d-c00da1ecb70e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:12.108755 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a602b-a45a-41c6-8f4d-c00da1ecb70e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:12.110381 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:12.121986 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 339/685] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:15:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:12.127978 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a602b-a45a-41c6-8f4d-c00da1ecb70e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:12.171190 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d437c55c-b483-42c4-a27d-96d0c8eb8ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:12.171797 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437c55c-b483-42c4-a27d-96d0c8eb8ee6 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:12.172071 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437c55c-b483-42c4-a27d-96d0c8eb8ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:12.172339 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437c55c-b483-42c4-a27d-96d0c8eb8ee6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:12.300688 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d437c55c-b483-42c4-a27d-96d0c8eb8ee6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:12.302155 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d437c55c-b483-42c4-a27d-96d0c8eb8ee6 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:12.311686 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a602b-a45a-41c6-8f4d-c00da1ecb70e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:12.362754 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024114ae-0f5e-4930-b4f2-79c4f39d066e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:12.363285 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024114ae-0f5e-4930-b4f2-79c4f39d066e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:12.363561 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024114ae-0f5e-4930-b4f2-79c4f39d066e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:12.363832 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024114ae-0f5e-4930-b4f2-79c4f39d066e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:12.364559 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024114ae-0f5e-4930-b4f2-79c4f39d066e None admin] RBAC: Adding query filter params (role.id=8ec32f702e0c447993ff294cbac536b1, scope.project.id=933820e2adbc473bbd57a4d59be75d72, user.id=5298486310004b63b51116dbc747e31d) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:12.364860 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024114ae-0f5e-4930-b4f2-79c4f39d066e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:12.367538 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024114ae-0f5e-4930-b4f2-79c4f39d066e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:12.376160 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 340/686] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:15:12 2022] GET /identity/v3/role_assignments?user.id=5298486310004b63b51116dbc747e31d&scope.project.id=933820e2adbc473bbd57a4d59be75d72&role.id=8ec32f702e0c447993ff294cbac536b1 => generated 608 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:12.377494 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024114ae-0f5e-4930-b4f2-79c4f39d066e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:12.461641 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:12.477114 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:12.497113 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 347/687] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:15:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:15:12.504364 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d437c55c-b483-42c4-a27d-96d0c8eb8ee6 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:12.522136 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024114ae-0f5e-4930-b4f2-79c4f39d066e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:12.565184 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22586e87-8718-489e-ae5d-be7595a2777c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:12.565942 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22586e87-8718-489e-ae5d-be7595a2777c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:12.566132 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22586e87-8718-489e-ae5d-be7595a2777c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:12.566407 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22586e87-8718-489e-ae5d-be7595a2777c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:12.581526 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22586e87-8718-489e-ae5d-be7595a2777c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:15:12.589141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-22586e87-8718-489e-ae5d-be7595a2777c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:15:12.589141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:12.589141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:12.589141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:12.589141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:12.589141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:12.589141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:12.589141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:12.589141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:12.589141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:12.589141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:12.589141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:12.589141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:15:12.589141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:15:12.589141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:15:12.589141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:15:12.590140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:15:12.590140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:15:12.590140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:15:12.590140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:15:12.590140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:12.590140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:12.590140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:12.590140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:12.590140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:12.590140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:12.590140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:12.590140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:12.590140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:12.590140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:12.590140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:12.590140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:12.590995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:12.590995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:12.590995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:12.590995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:12.590995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:15:12.590995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:15:12.590995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:15:12.590995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:15:12.590995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:15:12.590995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:15:12.590995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:15:12.590995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:12.590995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 341/688] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:15:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:12.597933 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22586e87-8718-489e-ae5d-be7595a2777c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:12.653546 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d437c55c-b483-42c4-a27d-96d0c8eb8ee6 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:12.704213 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df472ea0-afa9-49be-9701-f1b6410adc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:12.705396 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df472ea0-afa9-49be-9701-f1b6410adc80 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:12.705746 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df472ea0-afa9-49be-9701-f1b6410adc80 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:12.706239 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df472ea0-afa9-49be-9701-f1b6410adc80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:12.719436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df472ea0-afa9-49be-9701-f1b6410adc80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:12.721280 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df472ea0-afa9-49be-9701-f1b6410adc80 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:12.732306 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22586e87-8718-489e-ae5d-be7595a2777c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:12.742780 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-df472ea0-afa9-49be-9701-f1b6410adc80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:12.742780 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:12.742780 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:12.742780 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:12.742780 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:12.742780 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:12.742780 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:12.742780 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:12.742780 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:12.742780 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:12.742780 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:12.742780 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:12.742780 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:12.742780 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:12.742780 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:12.742780 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:12.743976 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:12.743976 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:12.743976 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:12.743976 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:12.743976 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:12.743976 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:12.743976 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:12.743976 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:12.743976 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:12.743976 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:12.743976 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:12.743976 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:12.743976 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:12.743976 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:12.743976 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:12.743976 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:12.743976 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:12.745140 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:12.745140 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:12.745140 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:12.745140 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 348/689] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:15:12.750185 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df472ea0-afa9-49be-9701-f1b6410adc80 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:12.763055 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:12.788873 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a3537-2be6-47e8-9983-5929ba853eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:12.790367 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a3537-2be6-47e8-9983-5929ba853eff None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:12.790840 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a3537-2be6-47e8-9983-5929ba853eff None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:12.791317 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a3537-2be6-47e8-9983-5929ba853eff None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:12.792169 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a3537-2be6-47e8-9983-5929ba853eff None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:12.792829 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a3537-2be6-47e8-9983-5929ba853eff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:12.795287 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a3537-2be6-47e8-9983-5929ba853eff None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:12.806783 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 342/690] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:15:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:12.806959 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:12.814612 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a3537-2be6-47e8-9983-5929ba853eff None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:12.875983 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df472ea0-afa9-49be-9701-f1b6410adc80 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:12.921498 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763156cd-15ca-4f89-b011-f35795b4b3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:12.922126 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763156cd-15ca-4f89-b011-f35795b4b3cc None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:12.922126 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763156cd-15ca-4f89-b011-f35795b4b3cc None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:12.922504 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763156cd-15ca-4f89-b011-f35795b4b3cc None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:12.923238 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763156cd-15ca-4f89-b011-f35795b4b3cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:12.923606 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763156cd-15ca-4f89-b011-f35795b4b3cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:12.925072 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763156cd-15ca-4f89-b011-f35795b4b3cc None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:12.940102 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 349/691] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:12.946359 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763156cd-15ca-4f89-b011-f35795b4b3cc None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:12.972024 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a3537-2be6-47e8-9983-5929ba853eff None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:13.038335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c95a6-e3c4-46c9-90dc-aa10b3df9636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:13.039084 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c95a6-e3c4-46c9-90dc-aa10b3df9636 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:13.039429 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c95a6-e3c4-46c9-90dc-aa10b3df9636 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:13.039768 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c95a6-e3c4-46c9-90dc-aa10b3df9636 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:13.055297 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18c95a6-e3c4-46c9-90dc-aa10b3df9636 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:13.056933 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18c95a6-e3c4-46c9-90dc-aa10b3df9636 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:13.068180 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763156cd-15ca-4f89-b011-f35795b4b3cc None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:13.071892 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:13.081961 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-c18c95a6-e3c4-46c9-90dc-aa10b3df9636 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:13.081961 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:13.081961 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:13.081961 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:13.081961 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:13.081961 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:13.081961 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:13.081961 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:13.081961 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:13.081961 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:13.081961 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:13.081961 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:13.081961 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:13.081961 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:13.081961 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:13.081961 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:13.082992 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:13.082992 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:13.082992 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:13.082992 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:13.082992 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:13.082992 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:13.082992 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:13.082992 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:13.082992 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:13.082992 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:13.082992 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:13.082992 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:13.082992 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:13.082992 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:13.082992 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:13.082992 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:13.082992 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:13.083925 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:13.083925 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:13.083925 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:13.086334 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 343/692] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:13.088500 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db5171-81e9-4d9d-bc07-9ef1571fb8e8 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:13.089059 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db5171-81e9-4d9d-bc07-9ef1571fb8e8 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:13.089523 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db5171-81e9-4d9d-bc07-9ef1571fb8e8 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:13.091916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 344/693] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:13.092785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28db5171-81e9-4d9d-bc07-9ef1571fb8e8 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:13.112662 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df94f0-997a-4b32-ac76-bb78e6236c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:13.113454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df94f0-997a-4b32-ac76-bb78e6236c15 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:13.113859 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df94f0-997a-4b32-ac76-bb78e6236c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:13.114222 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df94f0-997a-4b32-ac76-bb78e6236c15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:13.121890 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df94f0-997a-4b32-ac76-bb78e6236c15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:13.123459 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df94f0-997a-4b32-ac76-bb78e6236c15 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:13.135154 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:13.137517 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-d7df94f0-997a-4b32-ac76-bb78e6236c15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:13.137517 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:13.137517 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:13.137517 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:13.137517 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:13.137517 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:13.137517 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:13.137517 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:13.137517 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:13.137517 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:13.137517 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:13.137517 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:13.137517 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:15:13.137517 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:15:13.137517 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:15:13.138701 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:15:13.138701 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:13.138701 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:13.138701 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:13.138701 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:13.138701 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:13.138701 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:13.138701 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:13.138701 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:13.138701 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:13.138701 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:13.138701 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:13.138701 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:13.138701 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:13.138701 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:13.138701 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:13.138701 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:13.139772 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:15:13.139772 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:15:13.139772 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:15:13.139772 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:15:13.139772 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:15:13.139772 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:15:13.139772 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:13.139772 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:13.141123 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 350/694] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:15:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:13.142683 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d12bd-45ce-4942-899f-17e19dc6dfc4 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:13.143063 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d12bd-45ce-4942-899f-17e19dc6dfc4 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:13.143570 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d12bd-45ce-4942-899f-17e19dc6dfc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:13.164297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ff9d12bd-45ce-4942-899f-17e19dc6dfc4 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:13.165584 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ff9d12bd-45ce-4942-899f-17e19dc6dfc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:13.236565 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28db5171-81e9-4d9d-bc07-9ef1571fb8e8 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:13.239041 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 351/695] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:15:13.240143 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d12bd-45ce-4942-899f-17e19dc6dfc4 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:13.282467 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771deae-bee2-43c1-94e9-0fb457a2a382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:13.283184 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771deae-bee2-43c1-94e9-0fb457a2a382 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:13.283471 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771deae-bee2-43c1-94e9-0fb457a2a382 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:13.283747 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771deae-bee2-43c1-94e9-0fb457a2a382 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:13.284396 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f771deae-bee2-43c1-94e9-0fb457a2a382 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:13.284721 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f771deae-bee2-43c1-94e9-0fb457a2a382 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:13.286388 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f771deae-bee2-43c1-94e9-0fb457a2a382 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:13.300883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 345/696] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:13.306375 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771deae-bee2-43c1-94e9-0fb457a2a382 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:13.441638 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:13.457744 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d12bd-45ce-4942-899f-17e19dc6dfc4 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:13.468190 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:13.477951 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771deae-bee2-43c1-94e9-0fb457a2a382 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:13.517525 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa53127-fcda-494c-9484-3da6dd383eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:13.518164 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa53127-fcda-494c-9484-3da6dd383eda None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:13.518505 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa53127-fcda-494c-9484-3da6dd383eda None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:13.518841 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa53127-fcda-494c-9484-3da6dd383eda None admin] PATH_INFO: `/v3/users/nova` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:13.526797 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa53127-fcda-494c-9484-3da6dd383eda None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:13.528245 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa53127-fcda-494c-9484-3da6dd383eda None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:13.535445 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c6e84-deb7-4500-a005-fa6902530fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:13.536065 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c6e84-deb7-4500-a005-fa6902530fba None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:13.536347 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c6e84-deb7-4500-a005-fa6902530fba None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:13.536624 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c6e84-deb7-4500-a005-fa6902530fba None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:13.537398 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733c6e84-deb7-4500-a005-fa6902530fba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:13.537706 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733c6e84-deb7-4500-a005-fa6902530fba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:13.539188 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733c6e84-deb7-4500-a005-fa6902530fba None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:13.540184 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-faa53127-fcda-494c-9484-3da6dd383eda None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:15:13.540184 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:13.540184 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:13.540184 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:13.540184 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:13.540184 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:13.540184 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:13.540184 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:13.540184 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:13.540184 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:13.540184 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:13.540184 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:13.540184 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:15:13.540184 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:15:13.540184 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:15:13.540184 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:15:13.541105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:13.541105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:13.541105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:15:13.541105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:13.541105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:15:13.541105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:13.541105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:13.541105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:13.541105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:13.541105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:13.541105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:13.541105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:13.541105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:13.541105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:13.541105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:13.541105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:13.541105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:13.541105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:13.542125 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:13.542125 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:13.542125 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:15:13.542125 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:15:13.542125 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:15:13.542125 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:15:13.542125 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:15:13.542125 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:15:13.542125 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:15:13.542125 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:13.542125 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 346/697] 10.222.0.26 () {56 vars in 1093 bytes} [Fri Sep 9 20:15:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:13.542731 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa53127-fcda-494c-9484-3da6dd383eda None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:13.552165 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 352/698] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:15:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:13.553661 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c6e84-deb7-4500-a005-fa6902530fba None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:13.668147 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa53127-fcda-494c-9484-3da6dd383eda None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:13.673813 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c6e84-deb7-4500-a005-fa6902530fba None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:13.709936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24e406-1791-496d-aeb2-e2f783735914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:13.710592 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24e406-1791-496d-aeb2-e2f783735914 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:13.710871 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24e406-1791-496d-aeb2-e2f783735914 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:13.711143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24e406-1791-496d-aeb2-e2f783735914 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:13.711734 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24e406-1791-496d-aeb2-e2f783735914 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:13.713141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24e406-1791-496d-aeb2-e2f783735914 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:13.721440 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b9838-7d69-43a9-b11f-c511c0ecb154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:13.722243 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b9838-7d69-43a9-b11f-c511c0ecb154 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:13.722656 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b9838-7d69-43a9-b11f-c511c0ecb154 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:13.723023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b9838-7d69-43a9-b11f-c511c0ecb154 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:13.723905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468b9838-7d69-43a9-b11f-c511c0ecb154 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:13.724327 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468b9838-7d69-43a9-b11f-c511c0ecb154 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:13.726430 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468b9838-7d69-43a9-b11f-c511c0ecb154 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:13.726732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-af24e406-1791-496d-aeb2-e2f783735914 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 20:15:13.726732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:13.726732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:15:13.726732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:15:13.726732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:15:13.726732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:15:13.726732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:15:13.726732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:13.726732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 20:15:13.726732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:13.726732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:15:13.726732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:13.726732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:13.726732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:13.726732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:13.726732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:13.726732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:13.728030 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:13.728030 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:13.728030 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:13.728030 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:13.728030 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:13.728030 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:13.728030 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:15:13.728030 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:15:13.728030 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:15:13.728030 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:15:13.728030 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:13.728030 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:13.728030 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:13.728030 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:13.728030 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:13.728030 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:13.730119 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:13.730119 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:13.730119 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:13.730119 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:13.730119 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:13.730119 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:13.730119 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:13.730119 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:13.730119 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:13.730119 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:13.730119 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:15:13.730119 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:13.730119 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 20:15:13.730119 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:13.730119 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 347/699] 10.222.0.26 () {56 vars in 1129 bytes} [Fri Sep 9 20:15:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:13.731856 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24e406-1791-496d-aeb2-e2f783735914 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:13.748795 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:13.754079 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 353/700] 10.222.0.26 () {56 vars in 1133 bytes} [Fri Sep 9 20:15:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:13.755685 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b9838-7d69-43a9-b11f-c511c0ecb154 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:13.774300 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:13.840659 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24e406-1791-496d-aeb2-e2f783735914 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:13.875795 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de8579e-51bb-4fdc-a04d-4a3d38d56b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:13.876558 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de8579e-51bb-4fdc-a04d-4a3d38d56b6d None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:13.876940 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de8579e-51bb-4fdc-a04d-4a3d38d56b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:13.877333 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de8579e-51bb-4fdc-a04d-4a3d38d56b6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:13.878000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b9838-7d69-43a9-b11f-c511c0ecb154 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:13.878250 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de8579e-51bb-4fdc-a04d-4a3d38d56b6d None admin] RBAC: Adding query filter params (role.id=8ec32f702e0c447993ff294cbac536b1, scope.project.id=933820e2adbc473bbd57a4d59be75d72, user.id=fcb62d06f7e94e9295c54504ddabb1d4) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:13.878655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de8579e-51bb-4fdc-a04d-4a3d38d56b6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:13.880440 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de8579e-51bb-4fdc-a04d-4a3d38d56b6d None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:13.893278 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 348/701] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:15:13 2022] GET /identity/v3/role_assignments?user.id=fcb62d06f7e94e9295c54504ddabb1d4&scope.project.id=933820e2adbc473bbd57a4d59be75d72&role.id=8ec32f702e0c447993ff294cbac536b1 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:13.893278 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de8579e-51bb-4fdc-a04d-4a3d38d56b6d None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:13.954221 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e34ca40-a054-4443-a6b4-8d5b06ea35cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:13.954931 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34ca40-a054-4443-a6b4-8d5b06ea35cf None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:13.955131 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34ca40-a054-4443-a6b4-8d5b06ea35cf None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:13.955469 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34ca40-a054-4443-a6b4-8d5b06ea35cf None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:13.956182 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e34ca40-a054-4443-a6b4-8d5b06ea35cf None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:13.956486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e34ca40-a054-4443-a6b4-8d5b06ea35cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:13.960968 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e34ca40-a054-4443-a6b4-8d5b06ea35cf None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:13.966826 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 354/702] 10.222.0.26 () {56 vars in 1124 bytes} [Fri Sep 9 20:15:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:13.972098 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e34ca40-a054-4443-a6b4-8d5b06ea35cf None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:14.060651 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:14.083002 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de8579e-51bb-4fdc-a04d-4a3d38d56b6d None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:14.122896 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8427ca80-34ec-4537-acaf-b1ad77eb1917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:14.122896 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427ca80-34ec-4537-acaf-b1ad77eb1917 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:14.123625 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427ca80-34ec-4537-acaf-b1ad77eb1917 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:14.123734 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427ca80-34ec-4537-acaf-b1ad77eb1917 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:14.148831 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:14.177691 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e34ca40-a054-4443-a6b4-8d5b06ea35cf None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:14.201872 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8427ca80-34ec-4537-acaf-b1ad77eb1917 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:14.203654 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8427ca80-34ec-4537-acaf-b1ad77eb1917 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:14.222395 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a66af-c6ce-439e-a3b3-09e792456b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:14.223591 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a66af-c6ce-439e-a3b3-09e792456b46 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:14.224015 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a66af-c6ce-439e-a3b3-09e792456b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:14.224453 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a66af-c6ce-439e-a3b3-09e792456b46 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:14.225296 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361a66af-c6ce-439e-a3b3-09e792456b46 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:14.225740 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361a66af-c6ce-439e-a3b3-09e792456b46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:14.228071 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361a66af-c6ce-439e-a3b3-09e792456b46 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:14.235593 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 355/703] 10.222.0.26 () {56 vars in 1124 bytes} [Fri Sep 9 20:15:13 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:14.296072 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:14.322263 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 349/704] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:15:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:15:14.327908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a66af-c6ce-439e-a3b3-09e792456b46 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:14.448496 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a66af-c6ce-439e-a3b3-09e792456b46 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:14.483844 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db53371f-e4d4-482f-8b75-81d7feb20764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:14.484488 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db53371f-e4d4-482f-8b75-81d7feb20764 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:14.484764 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db53371f-e4d4-482f-8b75-81d7feb20764 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:14.485066 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db53371f-e4d4-482f-8b75-81d7feb20764 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:14.492484 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb502b0-1ab2-48dc-8fca-d530d56ee65c None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:14.492769 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb502b0-1ab2-48dc-8fca-d530d56ee65c None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:14.493051 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb502b0-1ab2-48dc-8fca-d530d56ee65c None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:14.494694 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 350/705] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:14.497027 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db53371f-e4d4-482f-8b75-81d7feb20764 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:14.498712 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db53371f-e4d4-482f-8b75-81d7feb20764 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:14.500486 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ec7f9-dfd6-4dad-b5e9-5a2192ba0bf9 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:14.500771 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ec7f9-dfd6-4dad-b5e9-5a2192ba0bf9 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:14.501049 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ec7f9-dfd6-4dad-b5e9-5a2192ba0bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:14.516630 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-db53371f-e4d4-482f-8b75-81d7feb20764 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:14.516630 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:14.516630 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:14.516630 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:14.516630 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:14.516630 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:14.516630 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:14.516630 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:14.516630 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:14.516630 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:14.516630 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:14.516630 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:14.516630 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:14.516630 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:14.516630 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:14.516630 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:14.517677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:14.517677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:14.517677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:14.517677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:14.517677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:14.517677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:14.517677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:14.517677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:14.517677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:14.517677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:14.517677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:14.517677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:14.517677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:14.517677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:14.517677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:14.517677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:14.517677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:14.518582 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:14.518582 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:14.518582 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:14.520898 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 356/706] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:14.525386 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:14.526604 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db53371f-e4d4-482f-8b75-81d7feb20764 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:14.550269 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-b44ec7f9-dfd6-4dad-b5e9-5a2192ba0bf9 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:14.551467 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-b44ec7f9-dfd6-4dad-b5e9-5a2192ba0bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:14.642512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db53371f-e4d4-482f-8b75-81d7feb20764 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:14.687627 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b2af93-5dd9-40b2-9c6b-97ae61c3593f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:14.688524 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2af93-5dd9-40b2-9c6b-97ae61c3593f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:14.688977 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2af93-5dd9-40b2-9c6b-97ae61c3593f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:14.689447 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2af93-5dd9-40b2-9c6b-97ae61c3593f None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:14.690433 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b2af93-5dd9-40b2-9c6b-97ae61c3593f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:14.690908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b2af93-5dd9-40b2-9c6b-97ae61c3593f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:14.692773 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b2af93-5dd9-40b2-9c6b-97ae61c3593f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:14.706612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 357/707] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:14.712423 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b2af93-5dd9-40b2-9c6b-97ae61c3593f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:14.747848 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 351/708] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:15:14.773449 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:14.824391 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b2af93-5dd9-40b2-9c6b-97ae61c3593f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:14.859754 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7abc383-3ae2-455f-9d5c-5751b495d6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:14.860774 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abc383-3ae2-455f-9d5c-5751b495d6cd None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:14.861300 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abc383-3ae2-455f-9d5c-5751b495d6cd None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:14.861808 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abc383-3ae2-455f-9d5c-5751b495d6cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:14.869574 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7abc383-3ae2-455f-9d5c-5751b495d6cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:14.871293 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7abc383-3ae2-455f-9d5c-5751b495d6cd None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:14.883553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-b7abc383-3ae2-455f-9d5c-5751b495d6cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:14.883553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:14.883553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:14.883553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:14.883553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:14.883553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:14.883553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:14.883553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:14.883553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:14.883553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:14.883553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:14.883553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:14.883553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:15:14.883553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:15:14.883553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:15:14.884531 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:15:14.884531 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:14.884531 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:14.884531 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:14.884531 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:14.884531 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:14.884531 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:14.884531 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:14.884531 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:14.884531 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:14.884531 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:14.884531 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:14.884531 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:14.884531 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:14.884531 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:14.884531 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:14.884531 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:14.885465 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:15:14.885465 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:15:14.885465 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:15:14.885465 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:15:14.885465 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:15:14.885465 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:15:14.885465 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:14.885465 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:14.887842 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 358/709] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:15:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:14.894382 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ec7f9-dfd6-4dad-b5e9-5a2192ba0bf9 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:15.041486 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ec7f9-dfd6-4dad-b5e9-5a2192ba0bf9 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:15.053366 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:15.073523 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7abc383-3ae2-455f-9d5c-5751b495d6cd None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:15.098876 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c4b9f-f3c6-46e3-9380-ed1c5c8384f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:15.099500 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c4b9f-f3c6-46e3-9380-ed1c5c8384f8 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:15.100340 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c4b9f-f3c6-46e3-9380-ed1c5c8384f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:15.100616 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c4b9f-f3c6-46e3-9380-ed1c5c8384f8 None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:15.101342 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95c4b9f-f3c6-46e3-9380-ed1c5c8384f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:15.101647 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95c4b9f-f3c6-46e3-9380-ed1c5c8384f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:15.103086 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95c4b9f-f3c6-46e3-9380-ed1c5c8384f8 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:15.116094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 352/710] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:15:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:15.121944 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c4b9f-f3c6-46e3-9380-ed1c5c8384f8 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:15.223894 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7abc383-3ae2-455f-9d5c-5751b495d6cd None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:15.244463 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:15.268123 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53cbf2-c313-40d4-8368-876d622e8596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:15.268779 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53cbf2-c313-40d4-8368-876d622e8596 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:15.269058 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53cbf2-c313-40d4-8368-876d622e8596 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:15.269361 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53cbf2-c313-40d4-8368-876d622e8596 None admin] PATH_INFO: `/v3/services/network` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:15.269970 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b53cbf2-c313-40d4-8368-876d622e8596 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:15.271836 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b53cbf2-c313-40d4-8368-876d622e8596 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:15.286275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-1b53cbf2-c313-40d4-8368-876d622e8596 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 20:15:15.286275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:15.286275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:15:15.286275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:15:15.286275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:15:15.286275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:15:15.286275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:15:15.286275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:15.286275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 20:15:15.286275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:15.286275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:15:15.286275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:15.286275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:15.286275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:15.286275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:15.286275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:15.286275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:15.287907 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:15.287907 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:15.287907 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:15.287907 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:15.287907 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:15.287907 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:15.287907 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:15:15.287907 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:15:15.287907 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:15:15.287907 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:15:15.287907 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:15.287907 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:15.287907 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:15.287907 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:15.287907 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:15.287907 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:15.288801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:15.288801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:15.288801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:15.288801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:15.288801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:15.288801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:15.288801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:15.288801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:15.288801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:15.288801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:15.288801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:15:15.288801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:15.288801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 20:15:15.288801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:15.288801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 359/711] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:15:15 2022] GET /identity/v3/services/network => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:15:15.289750 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a614e-a51d-49b9-92e6-7919edccebd6 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:15.290022 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a614e-a51d-49b9-92e6-7919edccebd6 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:15.290759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a614e-a51d-49b9-92e6-7919edccebd6 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:15.292346 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 360/712] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:15.293679 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a614e-a51d-49b9-92e6-7919edccebd6 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:15.297333 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c4b9f-f3c6-46e3-9380-ed1c5c8384f8 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:15.333982 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a4875-d728-4223-8931-e3975b5fb254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:15.334601 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a4875-d728-4223-8931-e3975b5fb254 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:15.334880 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a4875-d728-4223-8931-e3975b5fb254 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:15.335151 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a4875-d728-4223-8931-e3975b5fb254 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:15.335878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6a4875-d728-4223-8931-e3975b5fb254 None admin] RBAC: Adding query filter params (role.id=9121ba25a6d84756990f0c3a19e9fecb, scope.project.id=933820e2adbc473bbd57a4d59be75d72, user.id=87ece6128f9245f7b49f3e7a9faaed22) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:15.336181 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6a4875-d728-4223-8931-e3975b5fb254 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:15.338148 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6a4875-d728-4223-8931-e3975b5fb254 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:15.346957 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 353/713] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:15:15 2022] GET /identity/v3/role_assignments?user.id=87ece6128f9245f7b49f3e7a9faaed22&scope.project.id=933820e2adbc473bbd57a4d59be75d72&role.id=9121ba25a6d84756990f0c3a19e9fecb => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:15.348588 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe6e5f-9db7-45ea-a671-0fad1f3888cc None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:15.349051 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe6e5f-9db7-45ea-a671-0fad1f3888cc None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:15.349532 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe6e5f-9db7-45ea-a671-0fad1f3888cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:15.372318 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-defe6e5f-9db7-45ea-a671-0fad1f3888cc None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:15.374370 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-defe6e5f-9db7-45ea-a671-0fad1f3888cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:15.435725 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:15.465801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a614e-a51d-49b9-92e6-7919edccebd6 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:15.507101 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dbd176-2859-4c77-9814-2acfac7c8a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:15.508262 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dbd176-2859-4c77-9814-2acfac7c8a83 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:15.508781 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dbd176-2859-4c77-9814-2acfac7c8a83 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:15.509233 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dbd176-2859-4c77-9814-2acfac7c8a83 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:15.510102 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dbd176-2859-4c77-9814-2acfac7c8a83 None admin] RBAC: Adding query filter params (name=network) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:15.510581 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dbd176-2859-4c77-9814-2acfac7c8a83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:15.512253 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dbd176-2859-4c77-9814-2acfac7c8a83 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:15.512772 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 354/714] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:15:15.518035 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 361/715] 10.222.0.26 () {56 vars in 1112 bytes} [Fri Sep 9 20:15:15 2022] GET /identity/v3/services?name=network => generated 124 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:15.525935 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe6e5f-9db7-45ea-a671-0fad1f3888cc None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:15.658700 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe6e5f-9db7-45ea-a671-0fad1f3888cc None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:15.713016 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4650f8-b0e0-4b0e-b58f-ef0a915d7730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:15.713652 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4650f8-b0e0-4b0e-b58f-ef0a915d7730 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:15.713929 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4650f8-b0e0-4b0e-b58f-ef0a915d7730 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:15.714201 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4650f8-b0e0-4b0e-b58f-ef0a915d7730 None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:15.714826 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4650f8-b0e0-4b0e-b58f-ef0a915d7730 None admin] RBAC: Adding query filter params (type=network) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:15.715124 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4650f8-b0e0-4b0e-b58f-ef0a915d7730 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:15.716487 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4650f8-b0e0-4b0e-b58f-ef0a915d7730 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:15.724089 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 355/716] 10.222.0.26 () {56 vars in 1112 bytes} [Fri Sep 9 20:15:15 2022] GET /identity/v3/services?type=network => generated 124 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:15.989320 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dbd176-2859-4c77-9814-2acfac7c8a83 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:16.038177 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:16.097755 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dbd176-2859-4c77-9814-2acfac7c8a83 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:16.134277 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3014d9cb-5640-47e8-9c5c-68c5243d0184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:16.134910 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014d9cb-5640-47e8-9c5c-68c5243d0184 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:16.135184 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014d9cb-5640-47e8-9c5c-68c5243d0184 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:16.135450 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014d9cb-5640-47e8-9c5c-68c5243d0184 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:16.157660 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3014d9cb-5640-47e8-9c5c-68c5243d0184 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:16.159162 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3014d9cb-5640-47e8-9c5c-68c5243d0184 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:16.177185 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-3014d9cb-5640-47e8-9c5c-68c5243d0184 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:16.177185 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:16.177185 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:16.177185 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:16.177185 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:16.177185 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:16.177185 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:16.177185 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:16.177185 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:16.177185 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:16.177185 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:16.177185 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:16.177185 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:16.177185 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:16.177185 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:16.177185 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:16.178436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:16.178436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:16.178436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:16.178436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:16.178436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:16.178436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:16.178436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:16.178436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:16.178436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:16.178436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:16.178436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:16.178436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:16.178436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:16.178436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:16.178436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:16.178436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:16.178436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:16.179966 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:16.179966 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:16.179966 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:16.179966 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 362/717] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:16.184650 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4650f8-b0e0-4b0e-b58f-ef0a915d7730 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:16.238299 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:16.338278 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4650f8-b0e0-4b0e-b58f-ef0a915d7730 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:16.390009 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe23ec4-f128-4f44-8f27-ecdafcc52927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:16.390640 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe23ec4-f128-4f44-8f27-ecdafcc52927 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:16.390921 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe23ec4-f128-4f44-8f27-ecdafcc52927 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:16.391196 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe23ec4-f128-4f44-8f27-ecdafcc52927 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:16.391856 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe23ec4-f128-4f44-8f27-ecdafcc52927 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:16.392158 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe23ec4-f128-4f44-8f27-ecdafcc52927 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:16.393628 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe23ec4-f128-4f44-8f27-ecdafcc52927 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:16.406864 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 356/718] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:16.412371 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3014d9cb-5640-47e8-9c5c-68c5243d0184 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:16.540766 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:16.552331 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3014d9cb-5640-47e8-9c5c-68c5243d0184 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:16.605205 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37513504-cc89-416a-97ad-a614e7a5562a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:16.605818 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37513504-cc89-416a-97ad-a614e7a5562a None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:16.606101 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37513504-cc89-416a-97ad-a614e7a5562a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:16.606370 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37513504-cc89-416a-97ad-a614e7a5562a None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:16.607036 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37513504-cc89-416a-97ad-a614e7a5562a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:16.608453 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37513504-cc89-416a-97ad-a614e7a5562a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:16.641084 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 363/719] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Sep 9 20:15:16 2022] POST /identity/v3/users => generated 254 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:15:16.689054 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b8fc4e-bd4e-4e11-afe1-63d754fc19c1 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:16.689418 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b8fc4e-bd4e-4e11-afe1-63d754fc19c1 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:16.689635 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b8fc4e-bd4e-4e11-afe1-63d754fc19c1 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:16.691300 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 357/720] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:16.697808 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128dfe8e-a4ad-455a-8623-5b6908ab6b14 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:16.698099 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128dfe8e-a4ad-455a-8623-5b6908ab6b14 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:16.698378 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128dfe8e-a4ad-455a-8623-5b6908ab6b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:16.727587 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-128dfe8e-a4ad-455a-8623-5b6908ab6b14 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:16.728875 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-128dfe8e-a4ad-455a-8623-5b6908ab6b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:16.836098 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 364/721] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:15:16.904390 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353f159-ce72-4eae-abe7-9e9a0a749981 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:16.905025 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353f159-ce72-4eae-abe7-9e9a0a749981 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:16.905540 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353f159-ce72-4eae-abe7-9e9a0a749981 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:16.907416 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 358/722] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:16.913906 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff2692f-9df1-43d5-97dd-530f766a2acc None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:16.914193 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff2692f-9df1-43d5-97dd-530f766a2acc None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:16.914473 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff2692f-9df1-43d5-97dd-530f766a2acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:16.938355 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-7ff2692f-9df1-43d5-97dd-530f766a2acc None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:16.939541 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7ff2692f-9df1-43d5-97dd-530f766a2acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:17.018736 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 365/723] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:15:17.178368 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353f159-ce72-4eae-abe7-9e9a0a749981 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:17.276880 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:17.292690 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353f159-ce72-4eae-abe7-9e9a0a749981 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:17.329661 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff2692f-9df1-43d5-97dd-530f766a2acc None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:17.333571 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc37d4b-b05b-489c-a681-77a869cdbf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:17.334873 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc37d4b-b05b-489c-a681-77a869cdbf98 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:17.335158 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc37d4b-b05b-489c-a681-77a869cdbf98 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:17.335434 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc37d4b-b05b-489c-a681-77a869cdbf98 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:17.347228 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:17.347427 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc37d4b-b05b-489c-a681-77a869cdbf98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:15:17.353126 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-4bc37d4b-b05b-489c-a681-77a869cdbf98 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:15:17.353126 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:17.353126 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:17.353126 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:17.353126 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:17.353126 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:17.353126 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:17.353126 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:17.353126 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:17.353126 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:17.353126 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:17.353126 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:17.353126 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:15:17.353126 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:15:17.353126 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:15:17.353126 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:15:17.354132 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:15:17.354132 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:15:17.354132 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:15:17.354132 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:15:17.354132 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:17.354132 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:17.354132 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:17.354132 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:17.354132 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:17.354132 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:17.354132 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:17.354132 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:17.354132 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:17.354132 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:17.354132 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:17.354132 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:17.355021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:17.355021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:17.355021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:17.355021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:17.355021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:15:17.355021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:15:17.355021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:15:17.355021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:15:17.355021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:15:17.355021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:15:17.355021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:15:17.355021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:17.359434 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 359/724] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:15:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:17.360607 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc37d4b-b05b-489c-a681-77a869cdbf98 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:17.477046 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff2692f-9df1-43d5-97dd-530f766a2acc None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:17.515365 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37eddb0-c52d-4992-b2f4-c0a9e90d6fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:17.515365 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37eddb0-c52d-4992-b2f4-c0a9e90d6fa8 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:17.515365 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37eddb0-c52d-4992-b2f4-c0a9e90d6fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:17.515365 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37eddb0-c52d-4992-b2f4-c0a9e90d6fa8 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:17.515782 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37eddb0-c52d-4992-b2f4-c0a9e90d6fa8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:17.521281 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37eddb0-c52d-4992-b2f4-c0a9e90d6fa8 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:17.545848 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 366/725] 10.222.0.26 () {60 vars in 1141 bytes} [Fri Sep 9 20:15:17 2022] POST /identity/v3/services => generated 251 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:15:17.547047 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc37d4b-b05b-489c-a681-77a869cdbf98 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:17.614080 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7855d9f-a35b-4b30-b226-131c9c0c87ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:17.615109 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7855d9f-a35b-4b30-b226-131c9c0c87ae None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:17.615806 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7855d9f-a35b-4b30-b226-131c9c0c87ae None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:17.616286 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7855d9f-a35b-4b30-b226-131c9c0c87ae None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:17.617314 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7855d9f-a35b-4b30-b226-131c9c0c87ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:17.617820 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7855d9f-a35b-4b30-b226-131c9c0c87ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:17.619474 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7855d9f-a35b-4b30-b226-131c9c0c87ae None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:17.629312 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 360/726] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:15:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:17.636139 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37eddb0-c52d-4992-b2f4-c0a9e90d6fa8 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:17.754975 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:17.811439 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37eddb0-c52d-4992-b2f4-c0a9e90d6fa8 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:17.851466 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e9d4c6-970a-4775-914d-1f5b3dbe28c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:17.852150 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e9d4c6-970a-4775-914d-1f5b3dbe28c5 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:17.852425 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e9d4c6-970a-4775-914d-1f5b3dbe28c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:17.852693 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e9d4c6-970a-4775-914d-1f5b3dbe28c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:17.864269 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e9d4c6-970a-4775-914d-1f5b3dbe28c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:17.866280 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e9d4c6-970a-4775-914d-1f5b3dbe28c5 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:17.881182 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-28e9d4c6-970a-4775-914d-1f5b3dbe28c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:17.881182 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:17.881182 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:17.881182 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:17.881182 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:17.881182 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:17.881182 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:17.881182 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:17.881182 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:17.881182 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:17.881182 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:17.881182 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:17.881182 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:17.881182 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:17.881182 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:17.881182 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:17.882198 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:17.882198 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:17.882198 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:17.882198 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:17.882198 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:17.882198 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:17.882198 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:17.882198 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:17.882198 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:17.882198 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:17.882198 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:17.882198 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:17.882198 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:17.882198 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:17.882198 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:17.882198 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:17.882198 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:17.883288 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:17.883288 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:17.883288 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:17.883288 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 367/727] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:17.890102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7855d9f-a35b-4b30-b226-131c9c0c87ae None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:17.902350 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:18.045407 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7855d9f-a35b-4b30-b226-131c9c0c87ae None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:18.089878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0474f7d1-f381-4815-91a1-6a24e4c746ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:18.090483 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474f7d1-f381-4815-91a1-6a24e4c746ac None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:18.090796 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474f7d1-f381-4815-91a1-6a24e4c746ac None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:18.091073 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474f7d1-f381-4815-91a1-6a24e4c746ac None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:18.091709 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0474f7d1-f381-4815-91a1-6a24e4c746ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:18.092003 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0474f7d1-f381-4815-91a1-6a24e4c746ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:18.093381 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0474f7d1-f381-4815-91a1-6a24e4c746ac None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:18.106572 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 361/728] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:18.112195 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e9d4c6-970a-4775-914d-1f5b3dbe28c5 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:18.219070 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec698f3c-7427-473e-ab0b-84b21e85db08 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:18.219418 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec698f3c-7427-473e-ab0b-84b21e85db08 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:18.219627 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec698f3c-7427-473e-ab0b-84b21e85db08 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:18.220950 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 362/729] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:15:18.231216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018d119-d8df-45a4-ac43-a8b6fd4b8248 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:18.231843 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018d119-d8df-45a4-ac43-a8b6fd4b8248 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:18.232349 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018d119-d8df-45a4-ac43-a8b6fd4b8248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:18.265555 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-f018d119-d8df-45a4-ac43-a8b6fd4b8248 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:18.267096 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-f018d119-d8df-45a4-ac43-a8b6fd4b8248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:18.377723 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:18.393591 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 363/730] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:18 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:15:18.425942 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e9d4c6-970a-4775-914d-1f5b3dbe28c5 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:18.435517 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b9b7f-8e96-4540-8fb6-084f6880fe07 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:18.436165 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b9b7f-8e96-4540-8fb6-084f6880fe07 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:18.436639 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b9b7f-8e96-4540-8fb6-084f6880fe07 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:18.438193 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 364/731] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:18.442982 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d591376-df63-4d3f-b2bb-0ae66d6c0b07 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:18.443595 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d591376-df63-4d3f-b2bb-0ae66d6c0b07 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:18.444090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d591376-df63-4d3f-b2bb-0ae66d6c0b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:18.469317 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-0d591376-df63-4d3f-b2bb-0ae66d6c0b07 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:18.471652 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-0d591376-df63-4d3f-b2bb-0ae66d6c0b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:18.475452 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:18.502006 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75572ac3-87d9-43c5-99c2-3b6174590df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:18.503006 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75572ac3-87d9-43c5-99c2-3b6174590df7 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:18.503487 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75572ac3-87d9-43c5-99c2-3b6174590df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:18.504218 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75572ac3-87d9-43c5-99c2-3b6174590df7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:18.519124 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75572ac3-87d9-43c5-99c2-3b6174590df7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:18.521989 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75572ac3-87d9-43c5-99c2-3b6174590df7 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:18.549703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-75572ac3-87d9-43c5-99c2-3b6174590df7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 20:15:18.549703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:18.549703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:18.549703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:18.549703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:18.549703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:18.549703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:18.549703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:18.549703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:18.549703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:18.549703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:18.549703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:18.549703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:15:18.549703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:15:18.549703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:15:18.549703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:15:18.550702 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:18.550702 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:18.550702 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:15:18.550702 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:18.550702 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:15:18.550702 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:18.550702 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:18.550702 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:18.550702 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:18.550702 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:18.550702 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:18.550702 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:18.550702 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:18.550702 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:18.550702 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:18.550702 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:18.550702 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:18.550702 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:18.551692 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:18.551692 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:18.551692 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:15:18.551692 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:15:18.551692 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:15:18.551692 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:15:18.551692 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:15:18.551692 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:15:18.551692 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 20:15:18.551692 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:18.561288 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 368/732] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:15:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 445 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:18.563647 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75572ac3-87d9-43c5-99c2-3b6174590df7 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:18.604292 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 365/733] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:18 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:15:18.695493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75572ac3-87d9-43c5-99c2-3b6174590df7 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:18.713555 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d591376-df63-4d3f-b2bb-0ae66d6c0b07 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:18.733127 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3abfe10-5e8d-4fed-82ee-5332b86535ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:18.734066 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3abfe10-5e8d-4fed-82ee-5332b86535ae None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:18.734489 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3abfe10-5e8d-4fed-82ee-5332b86535ae None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:18.734904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3abfe10-5e8d-4fed-82ee-5332b86535ae None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:18.736241 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3abfe10-5e8d-4fed-82ee-5332b86535ae None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:18.737169 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3abfe10-5e8d-4fed-82ee-5332b86535ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:18.738758 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3abfe10-5e8d-4fed-82ee-5332b86535ae None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:18.763544 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 369/734] 10.222.0.26 () {56 vars in 1135 bytes} [Fri Sep 9 20:15:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:18.765759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3abfe10-5e8d-4fed-82ee-5332b86535ae None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:18.811258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:18.905641 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3abfe10-5e8d-4fed-82ee-5332b86535ae None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:18.924026 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d591376-df63-4d3f-b2bb-0ae66d6c0b07 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:18.967616 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:18.970978 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c753b20-3a68-4e9f-b7c7-b1a8034b8c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:18.972276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c753b20-3a68-4e9f-b7c7-b1a8034b8c3f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:18.972561 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c753b20-3a68-4e9f-b7c7-b1a8034b8c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:18.972832 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c753b20-3a68-4e9f-b7c7-b1a8034b8c3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:18.979530 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb42771a-b848-47d8-b75a-12b03c109130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:18.980249 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42771a-b848-47d8-b75a-12b03c109130 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:18.980598 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42771a-b848-47d8-b75a-12b03c109130 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:18.980947 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42771a-b848-47d8-b75a-12b03c109130 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:18.995088 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb42771a-b848-47d8-b75a-12b03c109130 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:15:19.002087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-cb42771a-b848-47d8-b75a-12b03c109130 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:15:19.002087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:19.002087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:19.002087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:19.002087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:19.002087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:19.002087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:19.002087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:19.002087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:19.002087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:19.002087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:19.002087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:19.002087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:15:19.002087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:15:19.002087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:15:19.002087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:15:19.003008 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:15:19.003008 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:15:19.003008 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:15:19.003008 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:15:19.003008 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:19.003008 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:19.003008 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:19.003008 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:19.003008 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:19.003008 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:19.003008 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:19.003008 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:19.003008 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:19.003008 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:19.003008 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:19.003008 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:19.004140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:19.004140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:19.004140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:19.004140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:19.004140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:15:19.004140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:15:19.004140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:15:19.004140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:15:19.004140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:15:19.004140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:15:19.004140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:15:19.004140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:19.005429 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 366/735] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:15:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:15:19.006855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb42771a-b848-47d8-b75a-12b03c109130 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:19.049780 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c753b20-3a68-4e9f-b7c7-b1a8034b8c3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:19.051212 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c753b20-3a68-4e9f-b7c7-b1a8034b8c3f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:19.093204 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:19.149507 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb42771a-b848-47d8-b75a-12b03c109130 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:19.163165 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 370/736] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:15:18 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 20:15:19.164291 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c753b20-3a68-4e9f-b7c7-b1a8034b8c3f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:19.197404 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a6975f-b9fe-4a3e-b325-b47413516116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:19.198514 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6975f-b9fe-4a3e-b325-b47413516116 None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:19.198946 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6975f-b9fe-4a3e-b325-b47413516116 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:19.199368 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6975f-b9fe-4a3e-b325-b47413516116 None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:19.200414 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a6975f-b9fe-4a3e-b325-b47413516116 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:19.202866 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a6975f-b9fe-4a3e-b325-b47413516116 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:19.224178 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 367/737] 10.222.0.26 () {60 vars in 1141 bytes} [Fri Sep 9 20:15:19 2022] POST /identity/v3/services => generated 240 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:15:19.225349 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a6975f-b9fe-4a3e-b325-b47413516116 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:19.286126 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:19.321086 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c753b20-3a68-4e9f-b7c7-b1a8034b8c3f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:19.413057 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a6975f-b9fe-4a3e-b325-b47413516116 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:19.426555 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41693aa7-7c0c-4fc5-8183-fc0aa405c517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:19.428031 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41693aa7-7c0c-4fc5-8183-fc0aa405c517 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:19.430579 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41693aa7-7c0c-4fc5-8183-fc0aa405c517 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:19.431109 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41693aa7-7c0c-4fc5-8183-fc0aa405c517 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:19.432009 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41693aa7-7c0c-4fc5-8183-fc0aa405c517 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:19.432781 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41693aa7-7c0c-4fc5-8183-fc0aa405c517 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:19.435633 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41693aa7-7c0c-4fc5-8183-fc0aa405c517 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:19.438519 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:19.455518 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 371/738] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:15:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:19.462082 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41693aa7-7c0c-4fc5-8183-fc0aa405c517 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:19.491364 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c0c035-f0fe-43ee-82ea-083000394fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:19.492018 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0c035-f0fe-43ee-82ea-083000394fee None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:19.492295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0c035-f0fe-43ee-82ea-083000394fee None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:19.492564 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0c035-f0fe-43ee-82ea-083000394fee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:19.508299 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c0c035-f0fe-43ee-82ea-083000394fee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:19.509820 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c0c035-f0fe-43ee-82ea-083000394fee None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:19.532746 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-51c0c035-f0fe-43ee-82ea-083000394fee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:19.532746 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:19.532746 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:19.532746 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:19.532746 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:19.532746 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:19.532746 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:19.532746 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:19.532746 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:19.532746 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:19.532746 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:19.532746 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:19.532746 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:19.532746 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:19.532746 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:19.532746 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:19.533784 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:19.533784 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:19.533784 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:19.533784 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:19.533784 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:19.533784 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:19.533784 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:19.533784 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:19.533784 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:19.533784 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:19.533784 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:19.533784 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:19.533784 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:19.533784 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:19.533784 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:19.533784 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:19.533784 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:19.534693 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:19.534693 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:19.534693 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:19.537345 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 368/739] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:19.540122 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c0c035-f0fe-43ee-82ea-083000394fee None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:19.583973 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41693aa7-7c0c-4fc5-8183-fc0aa405c517 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:19.627582 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816806a-16c1-46a8-ba95-7ee341b7e0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:19.629636 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816806a-16c1-46a8-ba95-7ee341b7e0ad None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:19.629919 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816806a-16c1-46a8-ba95-7ee341b7e0ad None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:19.630194 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816806a-16c1-46a8-ba95-7ee341b7e0ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:19.641895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7816806a-16c1-46a8-ba95-7ee341b7e0ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:19.643327 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7816806a-16c1-46a8-ba95-7ee341b7e0ad None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:19.658956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-7816806a-16c1-46a8-ba95-7ee341b7e0ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:19.658956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:19.658956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:19.658956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:19.658956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:19.658956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:19.658956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:19.658956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:19.658956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:19.658956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:19.658956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:19.658956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:19.658956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:19.658956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:19.658956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:19.658956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:19.660201 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:19.660201 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:19.660201 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:19.660201 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:19.660201 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:19.660201 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:19.660201 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:19.660201 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:19.660201 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:19.660201 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:19.660201 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:19.660201 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:19.660201 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:19.660201 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:19.660201 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:19.660201 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:19.660201 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:19.661430 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:19.661430 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:19.661430 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:19.661430 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 372/740] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:19.665750 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46f71d-d135-4055-97c2-d4a601d29a6f None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:19.666264 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46f71d-d135-4055-97c2-d4a601d29a6f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:19.666391 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46f71d-d135-4055-97c2-d4a601d29a6f None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:19.667821 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 373/741] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:19.669938 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c0c035-f0fe-43ee-82ea-083000394fee None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:19.673224 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8613c8-ace6-49ee-90d5-2ca707ab644e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:19.673518 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8613c8-ace6-49ee-90d5-2ca707ab644e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:19.673790 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8613c8-ace6-49ee-90d5-2ca707ab644e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:19.678092 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:19.694630 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ad8613c8-ace6-49ee-90d5-2ca707ab644e None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:19.695587 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ad8613c8-ace6-49ee-90d5-2ca707ab644e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:19.716480 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6fbaee-1889-4eb1-9b77-9a16baf7b09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:19.717888 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fbaee-1889-4eb1-9b77-9a16baf7b09a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:19.718495 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fbaee-1889-4eb1-9b77-9a16baf7b09a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:19.719218 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fbaee-1889-4eb1-9b77-9a16baf7b09a None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:19.720051 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6fbaee-1889-4eb1-9b77-9a16baf7b09a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:19.720810 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6fbaee-1889-4eb1-9b77-9a16baf7b09a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:19.722654 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6fbaee-1889-4eb1-9b77-9a16baf7b09a None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:19.741456 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 369/742] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:19.741456 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6fbaee-1889-4eb1-9b77-9a16baf7b09a None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:19.760218 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:19.797340 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 374/743] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:15:19.797340 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8613c8-ace6-49ee-90d5-2ca707ab644e None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:19.904631 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6fbaee-1889-4eb1-9b77-9a16baf7b09a None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:19.924441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8613c8-ace6-49ee-90d5-2ca707ab644e None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:19.975496 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f17de6-983d-4911-9b80-8e5b8fb90ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:19.976023 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f17de6-983d-4911-9b80-8e5b8fb90ccc None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:19.976299 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f17de6-983d-4911-9b80-8e5b8fb90ccc None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:19.976571 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f17de6-983d-4911-9b80-8e5b8fb90ccc None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:19.977212 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f17de6-983d-4911-9b80-8e5b8fb90ccc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:19.977540 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f17de6-983d-4911-9b80-8e5b8fb90ccc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:19.978902 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f17de6-983d-4911-9b80-8e5b8fb90ccc None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:19.993216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 370/744] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:19.998801 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f17de6-983d-4911-9b80-8e5b8fb90ccc None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:20.000227 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e277f-9cdb-4c0f-b788-e3b9bd6f13b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:20.000875 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e277f-9cdb-4c0f-b788-e3b9bd6f13b7 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:20.001148 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e277f-9cdb-4c0f-b788-e3b9bd6f13b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:20.001438 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e277f-9cdb-4c0f-b788-e3b9bd6f13b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:20.018209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963e277f-9cdb-4c0f-b788-e3b9bd6f13b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:20.021096 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963e277f-9cdb-4c0f-b788-e3b9bd6f13b7 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:20.032079 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:20.035590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-963e277f-9cdb-4c0f-b788-e3b9bd6f13b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:20.035590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:20.035590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:20.035590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:20.035590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:20.035590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:20.035590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:20.035590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:20.035590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:20.035590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:20.035590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:20.035590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:20.035590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:15:20.035590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:15:20.035590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:15:20.036545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:15:20.036545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:20.036545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:20.036545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:20.036545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:20.036545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:20.036545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:20.036545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:20.036545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:20.036545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:20.036545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:20.036545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:20.036545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:20.036545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:20.036545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:20.036545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:20.036545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:20.037678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:15:20.037678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:15:20.037678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:15:20.037678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:15:20.037678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:15:20.037678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:15:20.037678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:20.037678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:20.040916 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 375/745] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:15:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:20.046856 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e277f-9cdb-4c0f-b788-e3b9bd6f13b7 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:20.158502 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:20.188003 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f17de6-983d-4911-9b80-8e5b8fb90ccc None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:20.229730 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0087129-2351-41f1-881e-98541e2f59ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:20.230703 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0087129-2351-41f1-881e-98541e2f59ff None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:20.231208 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0087129-2351-41f1-881e-98541e2f59ff None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:20.231696 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0087129-2351-41f1-881e-98541e2f59ff None admin] PATH_INFO: `/v3/users/nova` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:20.231880 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e277f-9cdb-4c0f-b788-e3b9bd6f13b7 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:20.241308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0087129-2351-41f1-881e-98541e2f59ff None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:20.243436 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0087129-2351-41f1-881e-98541e2f59ff None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:20.256749 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-c0087129-2351-41f1-881e-98541e2f59ff None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:15:20.256749 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:20.256749 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:20.256749 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:20.256749 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:20.256749 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:20.256749 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:20.256749 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:20.256749 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:20.256749 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:20.256749 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:20.256749 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:20.256749 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:15:20.256749 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:15:20.256749 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:15:20.256749 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:15:20.257793 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:20.257793 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:20.257793 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:15:20.257793 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:20.257793 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:15:20.257793 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:20.257793 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:20.257793 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:20.257793 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:20.257793 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:20.257793 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:20.257793 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:20.257793 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:20.257793 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:20.257793 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:20.257793 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:20.257793 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:20.257793 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:20.258741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:20.258741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:20.258741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:15:20.258741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:15:20.258741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:15:20.258741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:15:20.258741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:15:20.258741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:15:20.258741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:15:20.258741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:20.258741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 371/746] 10.222.0.26 () {56 vars in 1093 bytes} [Fri Sep 9 20:15:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:20.258741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0087129-2351-41f1-881e-98541e2f59ff None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:20.274398 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b99e36-9a58-4b8e-badd-ca0ec0b85f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:20.275090 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b99e36-9a58-4b8e-badd-ca0ec0b85f78 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:20.275457 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b99e36-9a58-4b8e-badd-ca0ec0b85f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:20.275800 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b99e36-9a58-4b8e-badd-ca0ec0b85f78 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:20.276562 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b99e36-9a58-4b8e-badd-ca0ec0b85f78 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:20.276935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b99e36-9a58-4b8e-badd-ca0ec0b85f78 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:20.278384 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b99e36-9a58-4b8e-badd-ca0ec0b85f78 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:20.291033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 376/747] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:15:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:20.292612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2eebe6-f4cf-4a56-926a-bf6859ae04aa None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:20.293060 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2eebe6-f4cf-4a56-926a-bf6859ae04aa None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:20.293501 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2eebe6-f4cf-4a56-926a-bf6859ae04aa None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:20.295504 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 377/748] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:20.296665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2eebe6-f4cf-4a56-926a-bf6859ae04aa None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:20.343481 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:20.373060 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0087129-2351-41f1-881e-98541e2f59ff None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:20.405537 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:20.411256 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf793345-ff47-40f7-ba8c-126ce1309f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:20.411736 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2eebe6-f4cf-4a56-926a-bf6859ae04aa None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:20.412539 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf793345-ff47-40f7-ba8c-126ce1309f4a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:20.412709 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf793345-ff47-40f7-ba8c-126ce1309f4a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:20.412969 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf793345-ff47-40f7-ba8c-126ce1309f4a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:20.423462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf793345-ff47-40f7-ba8c-126ce1309f4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:15:20.429335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-bf793345-ff47-40f7-ba8c-126ce1309f4a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:15:20.429335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:20.429335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:20.429335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:20.429335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:20.429335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:20.429335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:20.429335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:20.429335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:20.429335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:20.429335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:20.429335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:20.429335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:15:20.429335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:15:20.429335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:15:20.429335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:15:20.430293 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:15:20.430293 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:15:20.430293 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:15:20.430293 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:15:20.430293 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:20.430293 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:20.430293 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:20.430293 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:20.430293 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:20.430293 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:20.430293 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:20.430293 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:20.430293 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:20.430293 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:20.430293 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:20.430293 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:20.431159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:20.431159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:20.431159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:20.431159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:20.431159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:15:20.431159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:15:20.431159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:15:20.431159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:15:20.431159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:15:20.431159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:15:20.431159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:15:20.431159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:20.431159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 372/749] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Sep 9 20:15:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:20.431874 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf793345-ff47-40f7-ba8c-126ce1309f4a None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:20.455035 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65191a9-7151-4c92-8794-4729367c40af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:20.455641 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65191a9-7151-4c92-8794-4729367c40af None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:20.455923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65191a9-7151-4c92-8794-4729367c40af None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:20.456195 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65191a9-7151-4c92-8794-4729367c40af None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:20.456919 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65191a9-7151-4c92-8794-4729367c40af None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:20.457224 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65191a9-7151-4c92-8794-4729367c40af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:20.458669 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65191a9-7151-4c92-8794-4729367c40af None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:20.481455 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 378/750] 10.222.0.26 () {56 vars in 1133 bytes} [Fri Sep 9 20:15:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:20.481455 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2cb81-0de1-4951-92dd-9e85b5ca03df None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:20.481455 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2cb81-0de1-4951-92dd-9e85b5ca03df None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:20.481455 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2cb81-0de1-4951-92dd-9e85b5ca03df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:20.495863 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-55b2cb81-0de1-4951-92dd-9e85b5ca03df None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:20.497036 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-55b2cb81-0de1-4951-92dd-9e85b5ca03df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:20.541672 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf793345-ff47-40f7-ba8c-126ce1309f4a None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:20.573273 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 379/751] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:15:20.573273 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b2cb81-0de1-4951-92dd-9e85b5ca03df None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:20.588681 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8703b-e606-43bd-94e4-10f1d3c57497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:20.589476 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8703b-e606-43bd-94e4-10f1d3c57497 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:20.590034 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8703b-e606-43bd-94e4-10f1d3c57497 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:20.590322 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8703b-e606-43bd-94e4-10f1d3c57497 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:20.591050 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced8703b-e606-43bd-94e4-10f1d3c57497 None admin] RBAC: Adding query filter params (role.id=9121ba25a6d84756990f0c3a19e9fecb, scope.project.id=933820e2adbc473bbd57a4d59be75d72, user.id=fcb62d06f7e94e9295c54504ddabb1d4) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:20.591360 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced8703b-e606-43bd-94e4-10f1d3c57497 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:20.593233 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced8703b-e606-43bd-94e4-10f1d3c57497 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:20.595479 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:20.605414 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 373/752] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:15:20 2022] GET /identity/v3/role_assignments?user.id=fcb62d06f7e94e9295c54504ddabb1d4&scope.project.id=933820e2adbc473bbd57a4d59be75d72&role.id=9121ba25a6d84756990f0c3a19e9fecb => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:20.606339 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8703b-e606-43bd-94e4-10f1d3c57497 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:20.716284 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:20.736667 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b2cb81-0de1-4951-92dd-9e85b5ca03df None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:20.775395 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d908f7-07d4-4b2c-8cb4-d847e5a79580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:20.776024 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d908f7-07d4-4b2c-8cb4-d847e5a79580 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:20.776305 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d908f7-07d4-4b2c-8cb4-d847e5a79580 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:20.776577 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d908f7-07d4-4b2c-8cb4-d847e5a79580 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:20.777271 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d908f7-07d4-4b2c-8cb4-d847e5a79580 None admin] RBAC: Adding query filter params (name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:20.777611 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d908f7-07d4-4b2c-8cb4-d847e5a79580 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:20.779008 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d908f7-07d4-4b2c-8cb4-d847e5a79580 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:20.788868 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 380/753] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Sep 9 20:15:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:20.794752 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d908f7-07d4-4b2c-8cb4-d847e5a79580 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:20.800292 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8703b-e606-43bd-94e4-10f1d3c57497 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:20.866016 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db48feb-5bb0-4a9e-8245-ca909a8d1e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:20.867079 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db48feb-5bb0-4a9e-8245-ca909a8d1e7f None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:20.867566 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db48feb-5bb0-4a9e-8245-ca909a8d1e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:20.868056 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db48feb-5bb0-4a9e-8245-ca909a8d1e7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:20.895279 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db48feb-5bb0-4a9e-8245-ca909a8d1e7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:20.897102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db48feb-5bb0-4a9e-8245-ca909a8d1e7f None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:20.903706 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d908f7-07d4-4b2c-8cb4-d847e5a79580 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:20.930882 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-9db48feb-5bb0-4a9e-8245-ca909a8d1e7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:20.930882 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:20.930882 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:20.930882 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:20.930882 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:20.930882 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:20.930882 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:20.930882 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:20.930882 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:20.930882 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:20.930882 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:20.930882 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:20.930882 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:20.930882 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:20.930882 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:20.930882 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:20.931889 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:20.931889 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:20.931889 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:20.931889 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:20.931889 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:20.931889 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:20.931889 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:20.931889 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:20.931889 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:20.931889 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:20.931889 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:20.931889 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:20.931889 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:20.931889 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:20.931889 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:20.931889 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:20.931889 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:20.932816 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:20.932816 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:20.932816 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:20.934911 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 374/754] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:20.940630 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db48feb-5bb0-4a9e-8245-ca909a8d1e7f None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:20.953740 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca29e8b-cb3b-4131-bee0-5878193626ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:20.954390 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca29e8b-cb3b-4131-bee0-5878193626ba None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:20.954671 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca29e8b-cb3b-4131-bee0-5878193626ba None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:20.955007 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca29e8b-cb3b-4131-bee0-5878193626ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:20.974885 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca29e8b-cb3b-4131-bee0-5878193626ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:20.976326 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca29e8b-cb3b-4131-bee0-5878193626ba None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:20.985429 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:20.994333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-dca29e8b-cb3b-4131-bee0-5878193626ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:20.994333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:20.994333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:20.994333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:20.994333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:20.994333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:20.994333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:20.994333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:20.994333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:20.994333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:20.994333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:20.994333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:20.994333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:20.994333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:20.994333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:20.994333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:20.995341 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:20.995341 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:20.995341 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:20.995341 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:20.995341 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:20.995341 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:20.995341 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:20.995341 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:20.995341 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:20.995341 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:20.995341 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:20.995341 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:20.995341 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:20.995341 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:20.995341 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:20.995341 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:20.995341 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:20.996277 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:20.996277 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:20.996277 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:20.997329 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 381/755] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:20.999205 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca29e8b-cb3b-4131-bee0-5878193626ba None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:21.004405 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:21.072682 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db48feb-5bb0-4a9e-8245-ca909a8d1e7f None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:21.117861 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca29e8b-cb3b-4131-bee0-5878193626ba None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:21.122271 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec1ee5a-9f63-4a6a-b3fa-f70f4133f406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:21.122884 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec1ee5a-9f63-4a6a-b3fa-f70f4133f406 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:21.123167 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec1ee5a-9f63-4a6a-b3fa-f70f4133f406 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:21.123439 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec1ee5a-9f63-4a6a-b3fa-f70f4133f406 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:21.124088 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec1ee5a-9f63-4a6a-b3fa-f70f4133f406 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:21.124390 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec1ee5a-9f63-4a6a-b3fa-f70f4133f406 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:21.125831 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec1ee5a-9f63-4a6a-b3fa-f70f4133f406 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:21.147524 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 375/756] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:21.147524 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec1ee5a-9f63-4a6a-b3fa-f70f4133f406 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:21.159831 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254e905-633a-4d53-9260-d010aa5b8ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:21.160642 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254e905-633a-4d53-9260-d010aa5b8ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:21.160831 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254e905-633a-4d53-9260-d010aa5b8ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:21.161106 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254e905-633a-4d53-9260-d010aa5b8ce5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:21.161736 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254e905-633a-4d53-9260-d010aa5b8ce5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:21.163812 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254e905-633a-4d53-9260-d010aa5b8ce5 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:21.174506 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-e254e905-633a-4d53-9260-d010aa5b8ce5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 20:15:21.174506 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:21.174506 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:15:21.174506 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:15:21.174506 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:15:21.174506 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:15:21.174506 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:15:21.174506 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:21.174506 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 20:15:21.174506 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:21.174506 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:15:21.174506 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:21.174506 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:21.174506 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:21.174506 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:21.174506 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:21.174506 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:21.175522 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:21.175522 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:21.175522 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:21.175522 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:21.175522 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:21.175522 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:21.175522 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:15:21.175522 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:15:21.175522 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:15:21.175522 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:15:21.175522 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:21.175522 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:21.175522 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:21.175522 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:21.175522 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:21.175522 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:21.176470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:21.176470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:21.176470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:21.176470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:21.176470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:21.176470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:21.176470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:21.176470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:21.176470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:21.176470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:21.176470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:15:21.176470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:21.176470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 20:15:21.176470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:21.180203 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 382/757] 10.222.0.26 () {56 vars in 1129 bytes} [Fri Sep 9 20:15:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:21.181339 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254e905-633a-4d53-9260-d010aa5b8ce5 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:21.255248 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:21.267123 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec1ee5a-9f63-4a6a-b3fa-f70f4133f406 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:21.281766 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:21.306880 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254e905-633a-4d53-9260-d010aa5b8ce5 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:21.315636 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de3935-aec9-46df-b7e2-fda71c805012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:21.316983 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de3935-aec9-46df-b7e2-fda71c805012 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:21.317530 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de3935-aec9-46df-b7e2-fda71c805012 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:21.318006 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de3935-aec9-46df-b7e2-fda71c805012 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:21.319719 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14de3935-aec9-46df-b7e2-fda71c805012 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:21.320530 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14de3935-aec9-46df-b7e2-fda71c805012 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:21.322466 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14de3935-aec9-46df-b7e2-fda71c805012 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:21.342406 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 376/758] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:21.342406 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de3935-aec9-46df-b7e2-fda71c805012 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:21.352382 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ec94d-afbe-43dc-9fb9-47d6b406a80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:21.353093 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ec94d-afbe-43dc-9fb9-47d6b406a80c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:21.353407 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ec94d-afbe-43dc-9fb9-47d6b406a80c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:21.353687 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ec94d-afbe-43dc-9fb9-47d6b406a80c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:21.362085 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ec94d-afbe-43dc-9fb9-47d6b406a80c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:21.364027 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ec94d-afbe-43dc-9fb9-47d6b406a80c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:21.378540 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-c54ec94d-afbe-43dc-9fb9-47d6b406a80c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:21.378540 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:21.378540 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:21.378540 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:21.378540 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:21.378540 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:21.378540 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:21.378540 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:21.378540 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:21.378540 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:21.378540 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:21.378540 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:21.378540 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:15:21.378540 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:15:21.378540 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:15:21.379753 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:15:21.379753 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:21.379753 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:21.379753 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:21.379753 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:21.379753 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:21.379753 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:21.379753 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:21.379753 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:21.379753 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:21.379753 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:21.379753 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:21.379753 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:21.379753 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:21.379753 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:21.379753 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:21.379753 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:21.380717 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:15:21.380717 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:15:21.380717 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:15:21.380717 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:15:21.380717 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:15:21.380717 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:15:21.380717 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:21.380717 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:21.384820 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 383/759] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:15:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:21.385750 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ec94d-afbe-43dc-9fb9-47d6b406a80c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:21.471358 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de3935-aec9-46df-b7e2-fda71c805012 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:21.507430 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ec94d-afbe-43dc-9fb9-47d6b406a80c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:21.520007 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c20da-a2cd-442d-af7e-d604ec82077f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:21.521878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c20da-a2cd-442d-af7e-d604ec82077f None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:21.522696 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c20da-a2cd-442d-af7e-d604ec82077f None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:21.523118 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c20da-a2cd-442d-af7e-d604ec82077f None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:21.523895 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645c20da-a2cd-442d-af7e-d604ec82077f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:21.524599 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645c20da-a2cd-442d-af7e-d604ec82077f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:21.526878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645c20da-a2cd-442d-af7e-d604ec82077f None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:21.537948 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 377/760] 10.222.0.26 () {56 vars in 1124 bytes} [Fri Sep 9 20:15:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:21.538179 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c20da-a2cd-442d-af7e-d604ec82077f None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:21.541859 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:21.557609 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86d7b5-0fd4-44db-ba08-6b8b7c47ca64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:21.558300 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86d7b5-0fd4-44db-ba08-6b8b7c47ca64 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:21.558585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86d7b5-0fd4-44db-ba08-6b8b7c47ca64 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:21.558862 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86d7b5-0fd4-44db-ba08-6b8b7c47ca64 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:21.567619 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86d7b5-0fd4-44db-ba08-6b8b7c47ca64 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:21.569844 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86d7b5-0fd4-44db-ba08-6b8b7c47ca64 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:21.582146 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:21.582383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-3b86d7b5-0fd4-44db-ba08-6b8b7c47ca64 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 20:15:21.582383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:21.582383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:21.582383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:21.582383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:21.582383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:21.582383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:21.582383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:21.582383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:21.582383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:21.582383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:21.582383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:21.582383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:15:21.582383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:15:21.582383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:15:21.582383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:15:21.583330 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:21.583330 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:21.583330 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:15:21.583330 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:21.583330 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:15:21.583330 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:21.583330 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:21.583330 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:21.583330 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:21.583330 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:21.583330 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:21.583330 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:21.583330 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:21.583330 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:21.583330 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:21.583330 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:21.583330 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:21.583330 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:21.584857 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:21.584857 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:21.584857 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:15:21.584857 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:15:21.584857 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:15:21.584857 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:15:21.584857 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:15:21.584857 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:15:21.584857 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 20:15:21.584857 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:21.587873 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 384/761] 10.222.0.26 () {56 vars in 1117 bytes} [Fri Sep 9 20:15:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:21.588728 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86d7b5-0fd4-44db-ba08-6b8b7c47ca64 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:21.669556 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c20da-a2cd-442d-af7e-d604ec82077f None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:21.709969 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cc5a12-b63e-4ae8-aa6d-3d7b1338cb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:21.710640 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc5a12-b63e-4ae8-aa6d-3d7b1338cb3c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:21.710883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc5a12-b63e-4ae8-aa6d-3d7b1338cb3c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:21.711167 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc5a12-b63e-4ae8-aa6d-3d7b1338cb3c None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:21.711849 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cc5a12-b63e-4ae8-aa6d-3d7b1338cb3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:21.712155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cc5a12-b63e-4ae8-aa6d-3d7b1338cb3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:21.713606 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cc5a12-b63e-4ae8-aa6d-3d7b1338cb3c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:21.725005 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86d7b5-0fd4-44db-ba08-6b8b7c47ca64 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:21.726113 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 378/762] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:15:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:21.731816 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cc5a12-b63e-4ae8-aa6d-3d7b1338cb3c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:21.765311 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5db73-856e-4867-8cd4-58603b778228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:21.766781 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5db73-856e-4867-8cd4-58603b778228 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:21.767214 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5db73-856e-4867-8cd4-58603b778228 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:21.767633 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5db73-856e-4867-8cd4-58603b778228 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:21.768424 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf5db73-856e-4867-8cd4-58603b778228 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:21.768977 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf5db73-856e-4867-8cd4-58603b778228 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:21.770743 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf5db73-856e-4867-8cd4-58603b778228 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:21.777498 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 385/763] 10.222.0.26 () {56 vars in 1124 bytes} [Fri Sep 9 20:15:21 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:21.778948 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5db73-856e-4867-8cd4-58603b778228 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:21.831178 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:21.833300 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:21.849074 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cc5a12-b63e-4ae8-aa6d-3d7b1338cb3c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:21.883516 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5db73-856e-4867-8cd4-58603b778228 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:21.884487 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8a4146-844d-43f6-8d89-68c7aa6f2c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:21.885099 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a4146-844d-43f6-8d89-68c7aa6f2c68 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:21.885413 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a4146-844d-43f6-8d89-68c7aa6f2c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:21.885682 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a4146-844d-43f6-8d89-68c7aa6f2c68 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:21.886424 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8a4146-844d-43f6-8d89-68c7aa6f2c68 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:21.886726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8a4146-844d-43f6-8d89-68c7aa6f2c68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:21.888150 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8a4146-844d-43f6-8d89-68c7aa6f2c68 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:21.910716 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 379/764] 10.222.0.26 () {56 vars in 1149 bytes} [Fri Sep 9 20:15:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:21.910716 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3b61d-b11f-4aac-b86d-5176172d7d90 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:21.910716 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3b61d-b11f-4aac-b86d-5176172d7d90 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:21.910716 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3b61d-b11f-4aac-b86d-5176172d7d90 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:21.910716 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 380/765] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:21.910716 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3b61d-b11f-4aac-b86d-5176172d7d90 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:21.927810 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0e88e-0047-4249-bc59-8c2a6c7772ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:21.928974 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0e88e-0047-4249-bc59-8c2a6c7772ae None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:21.929712 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0e88e-0047-4249-bc59-8c2a6c7772ae None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:21.930164 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0e88e-0047-4249-bc59-8c2a6c7772ae None admin] PATH_INFO: `/v3/projects/933820e2adbc473bbd57a4d59be75d72/users/87ece6128f9245f7b49f3e7a9faaed22/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:21.932591 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c0e88e-0047-4249-bc59-8c2a6c7772ae None admin] RBAC: Authorizing `identity:create_grant(project_id=933820e2adbc473bbd57a4d59be75d72, user_id=87ece6128f9245f7b49f3e7a9faaed22, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:21.934930 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c0e88e-0047-4249-bc59-8c2a6c7772ae None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:21.951424 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 386/766] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Sep 9 20:15:21 2022] PUT /identity/v3/projects/933820e2adbc473bbd57a4d59be75d72/users/87ece6128f9245f7b49f3e7a9faaed22/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:15:21.952490 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0e88e-0047-4249-bc59-8c2a6c7772ae None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:22.026403 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3b61d-b11f-4aac-b86d-5176172d7d90 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:22.071064 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f606f3-cd99-4431-b898-7cb4296e0914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:22.071064 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f606f3-cd99-4431-b898-7cb4296e0914 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:22.071348 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f606f3-cd99-4431-b898-7cb4296e0914 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:22.071515 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f606f3-cd99-4431-b898-7cb4296e0914 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:22.072187 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f606f3-cd99-4431-b898-7cb4296e0914 None admin] RBAC: Adding query filter params (interface=public, service_id=6afb2372feeb478fb2988444e15fc40c, region_id=RegionOne) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:22.072489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f606f3-cd99-4431-b898-7cb4296e0914 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:22.074126 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f606f3-cd99-4431-b898-7cb4296e0914 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:22.087722 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 381/767] 10.222.0.26 () {56 vars in 1251 bytes} [Fri Sep 9 20:15:21 2022] GET /identity/v3/endpoints?service_id=6afb2372feeb478fb2988444e15fc40c&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:22.089173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b868487-a506-4f7a-999f-3c623f2de926 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:22.089488 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b868487-a506-4f7a-999f-3c623f2de926 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:22.089762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b868487-a506-4f7a-999f-3c623f2de926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:22.109309 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-6b868487-a506-4f7a-999f-3c623f2de926 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:22.110524 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6b868487-a506-4f7a-999f-3c623f2de926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:22.111881 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0e88e-0047-4249-bc59-8c2a6c7772ae None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:22.123379 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:22.127024 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:22.166814 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c305dab9-2c51-4fcc-8829-88d2d1baa3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:22.167423 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c305dab9-2c51-4fcc-8829-88d2d1baa3d4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:22.167703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c305dab9-2c51-4fcc-8829-88d2d1baa3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:22.168024 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c305dab9-2c51-4fcc-8829-88d2d1baa3d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:22.197422 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 382/768] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:15:22.198291 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b868487-a506-4f7a-999f-3c623f2de926 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:22.286843 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c305dab9-2c51-4fcc-8829-88d2d1baa3d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:22.288251 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c305dab9-2c51-4fcc-8829-88d2d1baa3d4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:22.342175 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b868487-a506-4f7a-999f-3c623f2de926 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:22.414965 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 387/769] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:15:21 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:15:22.425134 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c305dab9-2c51-4fcc-8829-88d2d1baa3d4 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:22.435633 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea5fa7-32a8-404e-b450-c070c9927b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:22.436260 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea5fa7-32a8-404e-b450-c070c9927b81 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:22.436534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea5fa7-32a8-404e-b450-c070c9927b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:22.436804 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea5fa7-32a8-404e-b450-c070c9927b81 None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:22.437448 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ea5fa7-32a8-404e-b450-c070c9927b81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:22.438814 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ea5fa7-32a8-404e-b450-c070c9927b81 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:22.444130 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:22.454194 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 383/770] 10.222.0.26 () {56 vars in 1087 bytes} [Fri Sep 9 20:15:22 2022] GET /identity/v3/services => generated 772 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:22.552259 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c305dab9-2c51-4fcc-8829-88d2d1baa3d4 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:22.596935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1010b-5b80-4ecd-b9cd-4d2fd49d19e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:22.597622 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1010b-5b80-4ecd-b9cd-4d2fd49d19e8 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:22.597897 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1010b-5b80-4ecd-b9cd-4d2fd49d19e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:22.598162 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1010b-5b80-4ecd-b9cd-4d2fd49d19e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:22.601386 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea5fa7-32a8-404e-b450-c070c9927b81 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:22.610669 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc1010b-5b80-4ecd-b9cd-4d2fd49d19e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:22.612133 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc1010b-5b80-4ecd-b9cd-4d2fd49d19e8 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:22.626198 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:22.641628 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-ccc1010b-5b80-4ecd-b9cd-4d2fd49d19e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:22.641628 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:22.641628 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:22.641628 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:22.641628 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:22.641628 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:22.641628 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:22.641628 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:22.641628 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:22.641628 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:22.641628 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:22.641628 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:22.641628 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:22.641628 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:22.641628 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:22.641628 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:22.642678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:22.642678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:22.642678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:22.642678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:22.642678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:22.642678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:22.642678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:22.642678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:22.642678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:22.642678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:22.642678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:22.642678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:22.642678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:22.642678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:22.642678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:22.642678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:22.642678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:22.643585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:22.643585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:22.643585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:22.643585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 388/771] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:22.648521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1010b-5b80-4ecd-b9cd-4d2fd49d19e8 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:22.740520 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea5fa7-32a8-404e-b450-c070c9927b81 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:22.741127 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:22.764537 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1010b-5b80-4ecd-b9cd-4d2fd49d19e8 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:22.778911 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43dab2e-9f77-4446-bf37-ecc650998cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:22.779689 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43dab2e-9f77-4446-bf37-ecc650998cbf None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:22.780069 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43dab2e-9f77-4446-bf37-ecc650998cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:22.780443 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43dab2e-9f77-4446-bf37-ecc650998cbf None admin] PATH_INFO: `/v3/services/network` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:22.781103 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43dab2e-9f77-4446-bf37-ecc650998cbf None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:22.782562 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43dab2e-9f77-4446-bf37-ecc650998cbf None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:22.793036 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-d43dab2e-9f77-4446-bf37-ecc650998cbf None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 20:15:22.793036 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:22.793036 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:15:22.793036 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:15:22.793036 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:15:22.793036 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:15:22.793036 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:15:22.793036 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:22.793036 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 20:15:22.793036 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:22.793036 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:15:22.793036 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:22.793036 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:22.793036 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:22.793036 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:22.793036 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:22.793036 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:22.794094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:22.794094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:22.794094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:22.794094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:22.794094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:22.794094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:22.794094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:15:22.794094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:15:22.794094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:15:22.794094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:15:22.794094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:22.794094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:22.794094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:22.794094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:22.794094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:22.794094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:22.794945 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:22.794945 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:22.794945 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:22.794945 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:22.794945 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:22.794945 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:22.794945 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:22.794945 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:22.794945 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:22.794945 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:22.794945 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:15:22.794945 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:22.794945 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 20:15:22.794945 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:22.794945 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 384/772] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:15:22 2022] GET /identity/v3/services/network => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:15:22.804936 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a6b0f-15a3-4f12-b3a9-68c1c425a7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:22.805774 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a6b0f-15a3-4f12-b3a9-68c1c425a7c4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:22.806196 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a6b0f-15a3-4f12-b3a9-68c1c425a7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:22.806423 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43dab2e-9f77-4446-bf37-ecc650998cbf None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:22.806780 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a6b0f-15a3-4f12-b3a9-68c1c425a7c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:22.807580 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90a6b0f-15a3-4f12-b3a9-68c1c425a7c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:22.808017 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90a6b0f-15a3-4f12-b3a9-68c1c425a7c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:22.809675 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90a6b0f-15a3-4f12-b3a9-68c1c425a7c4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:22.823413 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 389/773] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:22.829108 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a6b0f-15a3-4f12-b3a9-68c1c425a7c4 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:22.904137 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:22.935695 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a6b0f-15a3-4f12-b3a9-68c1c425a7c4 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:22.936832 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43dab2e-9f77-4446-bf37-ecc650998cbf None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:22.959309 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:22.976420 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66c260-9a98-4b4a-8c18-6b5475c24f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:22.977457 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66c260-9a98-4b4a-8c18-6b5475c24f6b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:22.977938 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66c260-9a98-4b4a-8c18-6b5475c24f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:22.978404 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66c260-9a98-4b4a-8c18-6b5475c24f6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:22.979236 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d0dac7-8a88-4086-8be9-e5c4cc82b5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:22.979928 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0dac7-8a88-4086-8be9-e5c4cc82b5fa None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:22.980204 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0dac7-8a88-4086-8be9-e5c4cc82b5fa None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:22.980473 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0dac7-8a88-4086-8be9-e5c4cc82b5fa None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:22.981098 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d0dac7-8a88-4086-8be9-e5c4cc82b5fa None admin] RBAC: Adding query filter params (name=network) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:22.981705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d0dac7-8a88-4086-8be9-e5c4cc82b5fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:22.983462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d0dac7-8a88-4086-8be9-e5c4cc82b5fa None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:22.988104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b66c260-9a98-4b4a-8c18-6b5475c24f6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:22.988593 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 385/774] 10.222.0.26 () {56 vars in 1112 bytes} [Fri Sep 9 20:15:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:22.990487 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c931f14-14c1-4a25-85c8-56a1a1c227ee None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:22.990771 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c931f14-14c1-4a25-85c8-56a1a1c227ee None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:22.991042 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c931f14-14c1-4a25-85c8-56a1a1c227ee None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:22.991244 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b66c260-9a98-4b4a-8c18-6b5475c24f6b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:22.993322 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 386/775] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:22.994316 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c931f14-14c1-4a25-85c8-56a1a1c227ee None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:23.007543 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-8b66c260-9a98-4b4a-8c18-6b5475c24f6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:23.007543 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:23.007543 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:23.007543 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:23.007543 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:23.007543 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:23.007543 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:23.007543 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:23.007543 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:23.007543 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:23.007543 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:23.007543 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:23.007543 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:15:23.007543 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:15:23.007543 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:15:23.008486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:15:23.008486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:23.008486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:23.008486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:23.008486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:23.008486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:23.008486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:23.008486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:23.008486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:23.008486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:23.008486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:23.008486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:23.008486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:23.008486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:23.008486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:23.008486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:23.008486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:23.009424 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:15:23.009424 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:15:23.009424 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:15:23.009424 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:15:23.009424 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:15:23.009424 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:15:23.009424 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:23.009424 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:23.011874 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 390/776] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:15:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:15:23.017934 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95a3a5-c15e-4ca9-afa1-be09f0b3d052 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:23.018474 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95a3a5-c15e-4ca9-afa1-be09f0b3d052 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:23.018945 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95a3a5-c15e-4ca9-afa1-be09f0b3d052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:23.040881 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-8d95a3a5-c15e-4ca9-afa1-be09f0b3d052 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:23.042403 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8d95a3a5-c15e-4ca9-afa1-be09f0b3d052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:23.116007 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 391/777] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:15:23.117148 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95a3a5-c15e-4ca9-afa1-be09f0b3d052 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:23.137055 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c931f14-14c1-4a25-85c8-56a1a1c227ee None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:23.208450 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03748569-ee7d-405a-8d7d-1ae5e72a178f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:23.209652 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03748569-ee7d-405a-8d7d-1ae5e72a178f None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:23.209932 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03748569-ee7d-405a-8d7d-1ae5e72a178f None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:23.210205 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03748569-ee7d-405a-8d7d-1ae5e72a178f None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:23.210868 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03748569-ee7d-405a-8d7d-1ae5e72a178f None admin] RBAC: Adding query filter params (type=network) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:23.211131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03748569-ee7d-405a-8d7d-1ae5e72a178f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:23.212654 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03748569-ee7d-405a-8d7d-1ae5e72a178f None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:23.218255 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 387/778] 10.222.0.26 () {56 vars in 1112 bytes} [Fri Sep 9 20:15:22 2022] GET /identity/v3/services?type=network => generated 350 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:23.224187 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03748569-ee7d-405a-8d7d-1ae5e72a178f None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:23.233268 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:23.244839 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95a3a5-c15e-4ca9-afa1-be09f0b3d052 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:23.273173 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:23.334496 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0390ef-b4bc-4501-8e26-cad4b14ce65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:23.335689 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0390ef-b4bc-4501-8e26-cad4b14ce65a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:23.337402 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0390ef-b4bc-4501-8e26-cad4b14ce65a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:23.339043 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0390ef-b4bc-4501-8e26-cad4b14ce65a None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:23.339954 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0390ef-b4bc-4501-8e26-cad4b14ce65a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:23.340846 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0390ef-b4bc-4501-8e26-cad4b14ce65a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:23.342681 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0390ef-b4bc-4501-8e26-cad4b14ce65a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:23.364616 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03748569-ee7d-405a-8d7d-1ae5e72a178f None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:23.368854 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 392/779] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:15:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:23.375652 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0390ef-b4bc-4501-8e26-cad4b14ce65a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:23.437848 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec830cb-84b4-40e8-b853-2baeb35d9553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:23.438530 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec830cb-84b4-40e8-b853-2baeb35d9553 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:23.438813 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec830cb-84b4-40e8-b853-2baeb35d9553 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:23.439086 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec830cb-84b4-40e8-b853-2baeb35d9553 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:23.439796 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec830cb-84b4-40e8-b853-2baeb35d9553 None admin] RBAC: Adding query filter params (interface=public, service_id=84bdc9bd3a70415285490f967989d5bb, region_id=RegionOne) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:23.440103 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec830cb-84b4-40e8-b853-2baeb35d9553 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:23.441500 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec830cb-84b4-40e8-b853-2baeb35d9553 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:23.447936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 388/780] 10.222.0.26 () {56 vars in 1251 bytes} [Fri Sep 9 20:15:23 2022] GET /identity/v3/endpoints?service_id=84bdc9bd3a70415285490f967989d5bb&interface=public®ion_id=RegionOne => generated 194 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:23.453027 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec830cb-84b4-40e8-b853-2baeb35d9553 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:23.523598 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0390ef-b4bc-4501-8e26-cad4b14ce65a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:23.547038 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:23.562464 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c60c8-da29-4eb3-89e7-0a58f0242ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:23.563089 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c60c8-da29-4eb3-89e7-0a58f0242ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:23.563489 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c60c8-da29-4eb3-89e7-0a58f0242ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:23.563822 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c60c8-da29-4eb3-89e7-0a58f0242ef0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:23.564609 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962c60c8-da29-4eb3-89e7-0a58f0242ef0 None admin] RBAC: Adding query filter params (role.id=8ec32f702e0c447993ff294cbac536b1, scope.project.id=933820e2adbc473bbd57a4d59be75d72, user.id=cedaf156c5584d62beedde598c94f322) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:23.564980 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962c60c8-da29-4eb3-89e7-0a58f0242ef0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:23.566491 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962c60c8-da29-4eb3-89e7-0a58f0242ef0 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:23.572515 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 393/781] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:15:23 2022] GET /identity/v3/role_assignments?user.id=cedaf156c5584d62beedde598c94f322&scope.project.id=933820e2adbc473bbd57a4d59be75d72&role.id=8ec32f702e0c447993ff294cbac536b1 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:23.573515 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c60c8-da29-4eb3-89e7-0a58f0242ef0 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:23.601551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec830cb-84b4-40e8-b853-2baeb35d9553 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:23.612452 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:23.665174 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb38363-122b-4fc1-a286-9314f5c6f0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:23.665174 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb38363-122b-4fc1-a286-9314f5c6f0ef None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:23.665174 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb38363-122b-4fc1-a286-9314f5c6f0ef None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:23.665174 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb38363-122b-4fc1-a286-9314f5c6f0ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:23.696590 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb38363-122b-4fc1-a286-9314f5c6f0ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:15:23.703214 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-4bb38363-122b-4fc1-a286-9314f5c6f0ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:15:23.703214 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:23.703214 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:23.703214 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:23.703214 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:23.703214 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:23.703214 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:23.703214 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:23.703214 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:23.703214 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:23.703214 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:23.703214 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:23.703214 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:15:23.703214 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:15:23.703214 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:15:23.703214 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:15:23.704243 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:15:23.704243 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:15:23.704243 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:15:23.704243 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:15:23.704243 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:23.704243 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:23.704243 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:23.704243 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:23.704243 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:23.704243 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:23.704243 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:23.704243 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:23.704243 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:23.704243 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:23.704243 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:23.704243 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:23.705116 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:23.705116 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:23.705116 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:23.705116 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:23.705116 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:15:23.705116 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:15:23.705116 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:15:23.705116 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:15:23.705116 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:15:23.705116 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:15:23.705116 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:15:23.705116 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:23.713594 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 389/782] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:15:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:23.717041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb38363-122b-4fc1-a286-9314f5c6f0ef None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:23.771933 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c60c8-da29-4eb3-89e7-0a58f0242ef0 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:23.825567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d154cb03-c8b8-48b2-bd75-fa8e38d5f0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:23.826732 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154cb03-c8b8-48b2-bd75-fa8e38d5f0b9 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:23.826732 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154cb03-c8b8-48b2-bd75-fa8e38d5f0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:23.826909 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154cb03-c8b8-48b2-bd75-fa8e38d5f0b9 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:23.827396 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d154cb03-c8b8-48b2-bd75-fa8e38d5f0b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:23.828759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d154cb03-c8b8-48b2-bd75-fa8e38d5f0b9 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:23.836089 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 394/783] 10.222.0.26 () {56 vars in 1087 bytes} [Fri Sep 9 20:15:23 2022] GET /identity/v3/services => generated 772 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:23.867436 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb38363-122b-4fc1-a286-9314f5c6f0ef None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:23.897607 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:23.908832 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae69673-fb74-410a-a870-fa8e5a3cbb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:23.909932 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae69673-fb74-410a-a870-fa8e5a3cbb16 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:23.913698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae69673-fb74-410a-a870-fa8e5a3cbb16 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:23.914246 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae69673-fb74-410a-a870-fa8e5a3cbb16 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:23.915235 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae69673-fb74-410a-a870-fa8e5a3cbb16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:23.915782 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae69673-fb74-410a-a870-fa8e5a3cbb16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:23.917479 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae69673-fb74-410a-a870-fa8e5a3cbb16 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:23.934791 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 390/784] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:15:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:23.940694 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d154cb03-c8b8-48b2-bd75-fa8e38d5f0b9 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:24.077387 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:24.081794 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d154cb03-c8b8-48b2-bd75-fa8e38d5f0b9 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:24.129239 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4860e9a-1423-42c4-98b4-68640615f5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:24.129896 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4860e9a-1423-42c4-98b4-68640615f5ff None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:24.130169 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4860e9a-1423-42c4-98b4-68640615f5ff None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:24.130437 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4860e9a-1423-42c4-98b4-68640615f5ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:24.152294 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4860e9a-1423-42c4-98b4-68640615f5ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:24.152638 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4860e9a-1423-42c4-98b4-68640615f5ff None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:24.170336 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-e4860e9a-1423-42c4-98b4-68640615f5ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:24.170336 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:24.170336 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:24.170336 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:24.170336 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:24.170336 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:24.170336 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:24.170336 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:24.170336 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:24.170336 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:24.170336 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:24.170336 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:24.170336 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:24.170336 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:24.170336 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:24.170336 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:24.171332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:24.171332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:24.171332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:24.171332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:24.171332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:24.171332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:24.171332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:24.171332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:24.171332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:24.171332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:24.171332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:24.171332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:24.171332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:24.171332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:24.171332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:24.171332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:24.171332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:24.172915 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:24.172915 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:24.172915 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:24.172915 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 395/785] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:24.178150 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae69673-fb74-410a-a870-fa8e5a3cbb16 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:24.320246 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae69673-fb74-410a-a870-fa8e5a3cbb16 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:24.361148 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c068aa31-608e-4683-bdc0-b940ca192a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:24.362140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068aa31-608e-4683-bdc0-b940ca192a5b None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:24.362625 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068aa31-608e-4683-bdc0-b940ca192a5b None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:24.363123 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068aa31-608e-4683-bdc0-b940ca192a5b None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:24.364031 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c068aa31-608e-4683-bdc0-b940ca192a5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:24.364537 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c068aa31-608e-4683-bdc0-b940ca192a5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:24.366365 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c068aa31-608e-4683-bdc0-b940ca192a5b None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:24.379445 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 391/786] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:24.385159 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4860e9a-1423-42c4-98b4-68640615f5ff None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:24.403353 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf310d7-c085-44c8-bc10-ef2fb70d5c78 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:24.403637 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf310d7-c085-44c8-bc10-ef2fb70d5c78 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:24.403906 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf310d7-c085-44c8-bc10-ef2fb70d5c78 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:24.405504 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 392/787] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:15:24.410335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ee8da-b60b-4a53-9017-9d53e1ce5d0a None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:24.410661 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ee8da-b60b-4a53-9017-9d53e1ce5d0a None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:24.410938 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ee8da-b60b-4a53-9017-9d53e1ce5d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:24.429551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-af7ee8da-b60b-4a53-9017-9d53e1ce5d0a None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:24.430663 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-af7ee8da-b60b-4a53-9017-9d53e1ce5d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:24.487282 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:24.511720 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 393/788] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:15:24.599044 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4860e9a-1423-42c4-98b4-68640615f5ff None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:24.654505 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dbe63b-8b4f-4577-b169-479d2de0c7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:24.658058 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dbe63b-8b4f-4577-b169-479d2de0c7cb None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:24.658058 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dbe63b-8b4f-4577-b169-479d2de0c7cb None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:24.658058 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dbe63b-8b4f-4577-b169-479d2de0c7cb None admin] PATH_INFO: `/v3/users/swift` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:24.679125 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dbe63b-8b4f-4577-b169-479d2de0c7cb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:24.680631 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dbe63b-8b4f-4577-b169-479d2de0c7cb None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:24.698027 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-48dbe63b-8b4f-4577-b169-479d2de0c7cb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 20:15:24.698027 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:24.698027 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:24.698027 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:24.698027 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:24.698027 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:24.698027 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:24.698027 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:24.698027 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:24.698027 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:24.698027 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:24.698027 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:24.698027 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:15:24.698027 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:15:24.698027 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:15:24.698027 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:15:24.699078 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:24.699078 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:24.699078 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:15:24.699078 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:24.699078 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:15:24.699078 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:24.699078 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:24.699078 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:24.699078 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:24.699078 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:24.699078 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:24.699078 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:24.699078 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:24.699078 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:24.699078 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:24.699078 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:24.699078 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:24.699078 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:24.700695 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:24.700695 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:24.700695 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:15:24.700695 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:15:24.700695 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:15:24.700695 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:15:24.700695 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:15:24.700695 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:15:24.700695 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 20:15:24.700695 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:24.700695 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 396/789] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:15:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:24.707316 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ee8da-b60b-4a53-9017-9d53e1ce5d0a None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:24.858064 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dbe63b-8b4f-4577-b169-479d2de0c7cb None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:24.879049 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ee8da-b60b-4a53-9017-9d53e1ce5d0a None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:24.887191 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:24.918616 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513bcba1-def6-44ef-b635-b1266c733170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:24.919230 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513bcba1-def6-44ef-b635-b1266c733170 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:24.919506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513bcba1-def6-44ef-b635-b1266c733170 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:24.919772 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513bcba1-def6-44ef-b635-b1266c733170 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:24.920500 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513bcba1-def6-44ef-b635-b1266c733170 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:24.920799 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513bcba1-def6-44ef-b635-b1266c733170 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:24.922243 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513bcba1-def6-44ef-b635-b1266c733170 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:24.958005 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 394/790] 10.222.0.26 () {56 vars in 1135 bytes} [Fri Sep 9 20:15:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:24.965407 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513bcba1-def6-44ef-b635-b1266c733170 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:25.028832 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:25.036157 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dbe63b-8b4f-4577-b169-479d2de0c7cb None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:25.094143 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b498480-38c6-4bec-85af-00f2eab7880a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:25.094778 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b498480-38c6-4bec-85af-00f2eab7880a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:25.095111 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b498480-38c6-4bec-85af-00f2eab7880a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:25.095326 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b498480-38c6-4bec-85af-00f2eab7880a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:25.097196 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513bcba1-def6-44ef-b635-b1266c733170 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:25.124086 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b498480-38c6-4bec-85af-00f2eab7880a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:15:25.130123 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-6b498480-38c6-4bec-85af-00f2eab7880a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:15:25.130123 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:25.130123 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:25.130123 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:25.130123 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:25.130123 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:25.130123 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:25.130123 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:25.130123 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:25.130123 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:25.130123 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:25.130123 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:25.130123 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:15:25.130123 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:15:25.130123 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:15:25.130123 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:15:25.131786 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:15:25.131786 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:15:25.131786 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:15:25.131786 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:15:25.131786 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:25.131786 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:25.131786 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:25.131786 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:25.131786 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:25.131786 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:25.131786 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:25.131786 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:25.131786 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:25.131786 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:25.131786 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:25.131786 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:25.132698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:25.132698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:25.132698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:25.132698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:25.132698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:15:25.132698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:15:25.132698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:15:25.132698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:15:25.132698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:15:25.132698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:15:25.132698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:15:25.132698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:25.132698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 397/791] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:15:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:15:25.141444 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b498480-38c6-4bec-85af-00f2eab7880a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:25.143500 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d14a48c-0e62-4b09-a5d7-9fe8011eb89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:25.144182 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14a48c-0e62-4b09-a5d7-9fe8011eb89e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:25.144458 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14a48c-0e62-4b09-a5d7-9fe8011eb89e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:25.144725 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14a48c-0e62-4b09-a5d7-9fe8011eb89e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:25.165886 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d14a48c-0e62-4b09-a5d7-9fe8011eb89e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:25.167453 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d14a48c-0e62-4b09-a5d7-9fe8011eb89e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:25.193602 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-5d14a48c-0e62-4b09-a5d7-9fe8011eb89e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:25.193602 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:25.193602 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:25.193602 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:25.193602 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:25.193602 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:25.193602 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:25.193602 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:25.193602 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:25.193602 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:25.193602 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:25.193602 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:25.193602 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:25.193602 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:25.193602 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:25.193602 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:25.194762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:25.194762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:25.194762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:25.194762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:25.194762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:25.194762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:25.194762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:25.194762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:25.194762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:25.194762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:25.194762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:25.194762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:25.194762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:25.194762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:25.194762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:25.194762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:25.194762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:25.195695 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:25.195695 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:25.195695 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:25.195994 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 395/792] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:15:25.202256 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d14a48c-0e62-4b09-a5d7-9fe8011eb89e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:25.317375 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:25.325946 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:25.337798 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d14a48c-0e62-4b09-a5d7-9fe8011eb89e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:25.352536 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b498480-38c6-4bec-85af-00f2eab7880a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:25.372614 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a0c6e8-0d36-4e2b-90a7-d2881e8dea04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:25.373279 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0c6e8-0d36-4e2b-90a7-d2881e8dea04 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:25.373566 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0c6e8-0d36-4e2b-90a7-d2881e8dea04 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:25.373836 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0c6e8-0d36-4e2b-90a7-d2881e8dea04 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:25.374468 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a0c6e8-0d36-4e2b-90a7-d2881e8dea04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:25.374765 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a0c6e8-0d36-4e2b-90a7-d2881e8dea04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:25.376120 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a0c6e8-0d36-4e2b-90a7-d2881e8dea04 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:25.388679 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 396/793] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:25.388891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6643820a-1349-4041-933c-eb466460db17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:25.389546 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643820a-1349-4041-933c-eb466460db17 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:25.389833 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643820a-1349-4041-933c-eb466460db17 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:25.390112 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643820a-1349-4041-933c-eb466460db17 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:25.390216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5361c-d531-49e7-b7e8-b90fc1047224 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:25.390501 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5361c-d531-49e7-b7e8-b90fc1047224 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:25.390707 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5361c-d531-49e7-b7e8-b90fc1047224 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:25.390921 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6643820a-1349-4041-933c-eb466460db17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:25.391229 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6643820a-1349-4041-933c-eb466460db17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:25.392141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 397/794] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:25.392651 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6643820a-1349-4041-933c-eb466460db17 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:25.399630 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5361c-d531-49e7-b7e8-b90fc1047224 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:25.405652 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 398/795] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:15:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:25.407062 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae51a2-8f9b-4550-ae21-543bbe296b74 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:25.407354 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae51a2-8f9b-4550-ae21-543bbe296b74 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:25.407634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae51a2-8f9b-4550-ae21-543bbe296b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:25.426690 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-39ae51a2-8f9b-4550-ae21-543bbe296b74 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:25.427905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-39ae51a2-8f9b-4550-ae21-543bbe296b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:25.499528 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 399/796] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:15:25.500544 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ae51a2-8f9b-4550-ae21-543bbe296b74 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:25.538196 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5361c-d531-49e7-b7e8-b90fc1047224 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:25.602016 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6197c2d-006f-4942-8894-639b755312c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:25.602580 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6197c2d-006f-4942-8894-639b755312c4 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:25.602861 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6197c2d-006f-4942-8894-639b755312c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:25.603138 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6197c2d-006f-4942-8894-639b755312c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:25.612798 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:25.614448 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6197c2d-006f-4942-8894-639b755312c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:25.617270 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6197c2d-006f-4942-8894-639b755312c4 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:25.641396 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-f6197c2d-006f-4942-8894-639b755312c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:25.641396 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:25.641396 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:25.641396 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:25.641396 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:25.641396 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:25.641396 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:25.641396 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:25.641396 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:25.641396 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:25.641396 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:25.641396 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:25.641396 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:15:25.641396 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:15:25.641396 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:15:25.642329 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:15:25.642329 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:25.642329 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:25.642329 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:25.642329 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:25.642329 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:25.642329 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:25.642329 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:25.642329 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:25.642329 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:25.642329 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:25.642329 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:25.642329 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:25.642329 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:25.642329 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:25.642329 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:25.642329 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:25.643244 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:15:25.643244 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:15:25.643244 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:15:25.643244 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:15:25.643244 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:15:25.643244 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:15:25.643244 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:25.643244 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:25.644825 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 398/797] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:15:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:25.649480 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6197c2d-006f-4942-8894-639b755312c4 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:25.677653 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:25.698282 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ae51a2-8f9b-4550-ae21-543bbe296b74 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:25.776602 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6197c2d-006f-4942-8894-639b755312c4 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:25.781985 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a44bec9-dbd6-439f-ba74-258657505c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:25.782637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44bec9-dbd6-439f-ba74-258657505c4c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:25.782916 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44bec9-dbd6-439f-ba74-258657505c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:25.783186 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44bec9-dbd6-439f-ba74-258657505c4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:25.808023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a44bec9-dbd6-439f-ba74-258657505c4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:25.811558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a44bec9-dbd6-439f-ba74-258657505c4c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:25.824385 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f7f52-4eba-4169-b9bd-6f44c48bb6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:25.825233 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f7f52-4eba-4169-b9bd-6f44c48bb6c7 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:25.825738 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f7f52-4eba-4169-b9bd-6f44c48bb6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:25.826259 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f7f52-4eba-4169-b9bd-6f44c48bb6c7 None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:25.827174 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f7f52-4eba-4169-b9bd-6f44c48bb6c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:25.827659 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f7f52-4eba-4169-b9bd-6f44c48bb6c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:25.830369 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f7f52-4eba-4169-b9bd-6f44c48bb6c7 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:25.841953 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-1a44bec9-dbd6-439f-ba74-258657505c4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:25.841953 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:25.841953 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:25.841953 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:25.841953 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:25.841953 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:25.841953 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:25.841953 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:25.841953 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:25.841953 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:25.841953 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:25.841953 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:25.841953 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:25.841953 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:25.841953 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:25.841953 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:25.842939 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:25.842939 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:25.842939 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:25.842939 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:25.842939 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:25.842939 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:25.842939 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:25.842939 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:25.842939 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:25.842939 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:25.842939 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:25.842939 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:25.842939 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:25.842939 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:25.842939 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:25.842939 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:25.842939 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:25.843879 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:25.843879 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:25.843879 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:25.846801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 400/798] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:25.850618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a44bec9-dbd6-439f-ba74-258657505c4c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:25.854857 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 399/799] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:15:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:25.855786 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f7f52-4eba-4169-b9bd-6f44c48bb6c7 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:25.951526 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:25.968005 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a44bec9-dbd6-439f-ba74-258657505c4c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:25.985630 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:25.999605 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f7f52-4eba-4169-b9bd-6f44c48bb6c7 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:26.011582 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed699875-18a5-4f83-bb7f-573904992143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:26.012590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed699875-18a5-4f83-bb7f-573904992143 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:26.012880 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed699875-18a5-4f83-bb7f-573904992143 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:26.013159 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed699875-18a5-4f83-bb7f-573904992143 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:26.013860 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed699875-18a5-4f83-bb7f-573904992143 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:26.015914 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed699875-18a5-4f83-bb7f-573904992143 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:26.026894 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-ed699875-18a5-4f83-bb7f-573904992143 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 20:15:26.026894 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:26.026894 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:15:26.026894 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:15:26.026894 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:15:26.026894 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:15:26.026894 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:15:26.026894 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:26.026894 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 20:15:26.026894 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:26.026894 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:15:26.026894 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:26.026894 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:26.026894 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:26.026894 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:26.026894 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:26.026894 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:26.027951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:26.027951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:26.027951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:26.027951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:26.027951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:26.027951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:26.027951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:15:26.027951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:15:26.027951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:15:26.027951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:15:26.027951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:26.027951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:26.027951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:26.027951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:26.027951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:26.027951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:26.028818 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:26.028818 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:26.028818 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:26.028818 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:26.028818 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:26.028818 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:26.028818 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:26.028818 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:26.028818 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:26.028818 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:26.028818 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:15:26.028818 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:26.028818 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 20:15:26.028818 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:26.028818 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 401/800] 10.222.0.26 () {56 vars in 1129 bytes} [Fri Sep 9 20:15:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:26.029681 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed699875-18a5-4f83-bb7f-573904992143 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:26.051200 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e3705-d325-44a3-b542-4b7800a2ee40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:26.052966 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e3705-d325-44a3-b542-4b7800a2ee40 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:26.053462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e3705-d325-44a3-b542-4b7800a2ee40 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:26.053906 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e3705-d325-44a3-b542-4b7800a2ee40 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:26.055020 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e3705-d325-44a3-b542-4b7800a2ee40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:26.055973 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e3705-d325-44a3-b542-4b7800a2ee40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:26.057621 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e3705-d325-44a3-b542-4b7800a2ee40 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:26.078997 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 400/801] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:26.078997 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e3705-d325-44a3-b542-4b7800a2ee40 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:26.162123 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed699875-18a5-4f83-bb7f-573904992143 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:26.187550 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e3705-d325-44a3-b542-4b7800a2ee40 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:26.200519 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0788d75-be3d-4afb-bd71-93f684facf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:26.201167 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0788d75-be3d-4afb-bd71-93f684facf8e None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:26.201543 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0788d75-be3d-4afb-bd71-93f684facf8e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:26.201789 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0788d75-be3d-4afb-bd71-93f684facf8e None admin] PATH_INFO: `/v3/projects/933820e2adbc473bbd57a4d59be75d72/users/fcb62d06f7e94e9295c54504ddabb1d4/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:26.203780 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0788d75-be3d-4afb-bd71-93f684facf8e None admin] RBAC: Authorizing `identity:create_grant(project_id=933820e2adbc473bbd57a4d59be75d72, user_id=fcb62d06f7e94e9295c54504ddabb1d4, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:26.205229 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0788d75-be3d-4afb-bd71-93f684facf8e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:26.220720 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:26.221609 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 402/802] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Sep 9 20:15:26 2022] PUT /identity/v3/projects/933820e2adbc473bbd57a4d59be75d72/users/fcb62d06f7e94e9295c54504ddabb1d4/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:15:26.222249 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0788d75-be3d-4afb-bd71-93f684facf8e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:26.242654 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63df3f04-6c13-46c9-8259-8638a61d7713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:26.244602 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df3f04-6c13-46c9-8259-8638a61d7713 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:26.245171 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df3f04-6c13-46c9-8259-8638a61d7713 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:26.245915 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df3f04-6c13-46c9-8259-8638a61d7713 None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:26.247343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63df3f04-6c13-46c9-8259-8638a61d7713 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:26.247991 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63df3f04-6c13-46c9-8259-8638a61d7713 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:26.250449 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63df3f04-6c13-46c9-8259-8638a61d7713 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:26.259713 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 401/803] 10.222.0.26 () {56 vars in 1124 bytes} [Fri Sep 9 20:15:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:26.262326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593c4d9-082d-420e-9da9-f25c3f9a0144 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:26.262919 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593c4d9-082d-420e-9da9-f25c3f9a0144 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:26.263500 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593c4d9-082d-420e-9da9-f25c3f9a0144 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:26.266643 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 402/804] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:26.267931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593c4d9-082d-420e-9da9-f25c3f9a0144 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:26.270739 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:26.347308 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0788d75-be3d-4afb-bd71-93f684facf8e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:26.394969 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f73eb-89a0-4427-a920-17066d37a155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:26.395755 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f73eb-89a0-4427-a920-17066d37a155 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:26.396137 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f73eb-89a0-4427-a920-17066d37a155 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:26.396483 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f73eb-89a0-4427-a920-17066d37a155 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:26.409042 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5f73eb-89a0-4427-a920-17066d37a155 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:26.413798 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5f73eb-89a0-4427-a920-17066d37a155 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:26.428928 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593c4d9-082d-420e-9da9-f25c3f9a0144 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:26.431285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-8c5f73eb-89a0-4427-a920-17066d37a155 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:15:26.431285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:26.431285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:26.431285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:26.431285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:26.431285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:26.431285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:26.431285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:26.431285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:26.431285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:26.431285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:26.431285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:26.431285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:15:26.431285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:15:26.431285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:15:26.431285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:15:26.432297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:26.432297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:26.432297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:15:26.432297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:26.432297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:15:26.432297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:26.432297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:26.432297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:26.432297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:26.432297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:26.432297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:26.432297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:26.432297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:26.432297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:26.432297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:26.432297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:26.432297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:26.432297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:26.433300 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:26.433300 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:26.433300 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:15:26.433300 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:15:26.433300 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:15:26.433300 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:15:26.433300 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:15:26.433300 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:15:26.433300 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:15:26.433300 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:26.435041 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 403/805] 10.222.0.26 () {56 vars in 1093 bytes} [Fri Sep 9 20:15:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:26.436342 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862077b4-cfa8-4b6a-a838-2ff225308c41 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:26.436632 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862077b4-cfa8-4b6a-a838-2ff225308c41 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:26.436909 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862077b4-cfa8-4b6a-a838-2ff225308c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:26.457813 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-862077b4-cfa8-4b6a-a838-2ff225308c41 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:26.459068 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-862077b4-cfa8-4b6a-a838-2ff225308c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:26.471770 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ceb57-05dd-416e-b9a3-832f9eb6fd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:26.472375 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ceb57-05dd-416e-b9a3-832f9eb6fd17 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:26.472651 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ceb57-05dd-416e-b9a3-832f9eb6fd17 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:26.472920 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ceb57-05dd-416e-b9a3-832f9eb6fd17 None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:26.473602 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0ceb57-05dd-416e-b9a3-832f9eb6fd17 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:26.473904 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0ceb57-05dd-416e-b9a3-832f9eb6fd17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:26.475901 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0ceb57-05dd-416e-b9a3-832f9eb6fd17 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:26.481175 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 403/806] 10.222.0.26 () {56 vars in 1124 bytes} [Fri Sep 9 20:15:26 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:26.482004 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ceb57-05dd-416e-b9a3-832f9eb6fd17 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:26.545494 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 404/807] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:15:26.546650 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862077b4-cfa8-4b6a-a838-2ff225308c41 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:26.558235 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:26.577189 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:26.617061 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ceb57-05dd-416e-b9a3-832f9eb6fd17 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:26.652971 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d049e-2242-4531-8804-97163a1dcc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:26.653634 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d049e-2242-4531-8804-97163a1dcc69 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:26.653920 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d049e-2242-4531-8804-97163a1dcc69 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:26.654200 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d049e-2242-4531-8804-97163a1dcc69 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:26.654971 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8d049e-2242-4531-8804-97163a1dcc69 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:26.655242 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8d049e-2242-4531-8804-97163a1dcc69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:26.656684 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8d049e-2242-4531-8804-97163a1dcc69 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:26.659478 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862077b4-cfa8-4b6a-a838-2ff225308c41 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:26.679287 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 404/808] 10.222.0.26 () {56 vars in 1133 bytes} [Fri Sep 9 20:15:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:26.679287 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de575e8c-dd7b-4cde-a394-222eb0ab06b6 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:26.679287 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de575e8c-dd7b-4cde-a394-222eb0ab06b6 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:26.679287 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de575e8c-dd7b-4cde-a394-222eb0ab06b6 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:26.682739 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 405/809] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:26.683850 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69b116-30f4-416a-bf89-1330523efb48 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:26.684229 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69b116-30f4-416a-bf89-1330523efb48 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:26.684415 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69b116-30f4-416a-bf89-1330523efb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:26.705289 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-db69b116-30f4-416a-bf89-1330523efb48 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:26.705289 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-db69b116-30f4-416a-bf89-1330523efb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:26.713783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5919a685-e288-43da-9939-8da41db9b67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:26.714609 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919a685-e288-43da-9939-8da41db9b67e None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:26.715094 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919a685-e288-43da-9939-8da41db9b67e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:26.715550 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919a685-e288-43da-9939-8da41db9b67e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:26.716399 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5919a685-e288-43da-9939-8da41db9b67e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:26.717957 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5919a685-e288-43da-9939-8da41db9b67e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:26.744753 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 405/810] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:15:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:15:26.746552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5919a685-e288-43da-9939-8da41db9b67e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:26.807766 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 406/811] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:15:26.850532 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db69b116-30f4-416a-bf89-1330523efb48 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:26.916706 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:26.916909 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:26.948319 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5919a685-e288-43da-9939-8da41db9b67e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:27.008841 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fddb15-14cc-4bb9-aebe-8218ebe52dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:27.010112 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fddb15-14cc-4bb9-aebe-8218ebe52dd5 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:27.010581 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fddb15-14cc-4bb9-aebe-8218ebe52dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:27.011062 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fddb15-14cc-4bb9-aebe-8218ebe52dd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:27.015739 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db69b116-30f4-416a-bf89-1330523efb48 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:27.066410 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0e570c-4580-4f7a-bef2-e3f30056d022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:27.067323 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e570c-4580-4f7a-bef2-e3f30056d022 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:27.067748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e570c-4580-4f7a-bef2-e3f30056d022 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:27.069152 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e570c-4580-4f7a-bef2-e3f30056d022 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:27.084854 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0e570c-4580-4f7a-bef2-e3f30056d022 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:15:27.092146 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-ba0e570c-4580-4f7a-bef2-e3f30056d022 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:15:27.092146 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:27.092146 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:27.092146 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:27.092146 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:27.092146 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:27.092146 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:27.092146 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:27.092146 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:27.092146 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:27.092146 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:27.092146 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:27.092146 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:15:27.092146 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:15:27.092146 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:15:27.092146 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:15:27.094090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:15:27.094090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:15:27.094090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:15:27.094090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:15:27.094090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:27.094090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:27.094090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:27.094090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:27.094090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:27.094090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:27.094090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:27.094090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:27.094090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:27.094090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:27.094090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:27.094090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:27.095094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:27.095094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:27.095094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:27.095094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:27.095094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:15:27.095094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:15:27.095094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:15:27.095094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:15:27.095094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:15:27.095094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:15:27.095094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:15:27.095094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:27.097654 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 407/812] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Sep 9 20:15:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:15:27.104163 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0e570c-4580-4f7a-bef2-e3f30056d022 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:27.109412 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fddb15-14cc-4bb9-aebe-8218ebe52dd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:27.111747 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fddb15-14cc-4bb9-aebe-8218ebe52dd5 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:27.206814 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:27.221693 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 406/813] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:15:26 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:15:27.222893 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fddb15-14cc-4bb9-aebe-8218ebe52dd5 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:27.242475 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:27.245793 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0e570c-4580-4f7a-bef2-e3f30056d022 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:27.293961 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039d1d6-40f0-4a88-9dfb-70f5bebf6c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:27.294580 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039d1d6-40f0-4a88-9dfb-70f5bebf6c88 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:27.294862 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039d1d6-40f0-4a88-9dfb-70f5bebf6c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:27.295138 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039d1d6-40f0-4a88-9dfb-70f5bebf6c88 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:27.296057 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7039d1d6-40f0-4a88-9dfb-70f5bebf6c88 None admin] RBAC: Adding query filter params (name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:27.296372 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7039d1d6-40f0-4a88-9dfb-70f5bebf6c88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:27.298476 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7039d1d6-40f0-4a88-9dfb-70f5bebf6c88 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:27.309884 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 408/814] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Sep 9 20:15:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:27.310721 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039d1d6-40f0-4a88-9dfb-70f5bebf6c88 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:27.346220 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fddb15-14cc-4bb9-aebe-8218ebe52dd5 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:27.384925 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78caadf9-338a-42f0-abdd-325dc2e68363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:27.385605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78caadf9-338a-42f0-abdd-325dc2e68363 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:27.385887 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78caadf9-338a-42f0-abdd-325dc2e68363 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:27.386164 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78caadf9-338a-42f0-abdd-325dc2e68363 None admin] PATH_INFO: `/v3/services/network` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:27.386741 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78caadf9-338a-42f0-abdd-325dc2e68363 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:27.388101 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78caadf9-338a-42f0-abdd-325dc2e68363 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:27.400222 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-78caadf9-338a-42f0-abdd-325dc2e68363 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 20:15:27.400222 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:27.400222 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:15:27.400222 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:15:27.400222 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:15:27.400222 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:15:27.400222 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:15:27.400222 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:27.400222 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 20:15:27.400222 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:27.400222 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:15:27.400222 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:27.400222 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:27.400222 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:27.400222 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:27.400222 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:27.400222 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:27.401286 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:27.401286 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:27.401286 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:27.401286 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:27.401286 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:27.401286 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:27.401286 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:15:27.401286 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:15:27.401286 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:15:27.401286 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:15:27.401286 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:27.401286 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:27.401286 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:27.401286 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:27.401286 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:27.401286 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:27.402161 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:27.402161 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:27.402161 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:27.402161 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:27.402161 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:27.402161 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:27.402161 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:27.402161 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:27.402161 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:27.402161 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:27.402161 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:15:27.402161 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:27.402161 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 20:15:27.402161 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:27.406247 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 407/815] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:15:27 2022] GET /identity/v3/services/network => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:27.407077 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78caadf9-338a-42f0-abdd-325dc2e68363 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:27.454226 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039d1d6-40f0-4a88-9dfb-70f5bebf6c88 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:27.476649 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:27.510541 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8a251c-9189-41c3-be57-07401119e693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:27.511219 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a251c-9189-41c3-be57-07401119e693 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:27.511501 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a251c-9189-41c3-be57-07401119e693 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:27.511822 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a251c-9189-41c3-be57-07401119e693 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:27.523588 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8a251c-9189-41c3-be57-07401119e693 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:27.525202 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8a251c-9189-41c3-be57-07401119e693 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:27.534521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78caadf9-338a-42f0-abdd-325dc2e68363 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:27.542807 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-0c8a251c-9189-41c3-be57-07401119e693 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:27.542807 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:27.542807 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:27.542807 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:27.542807 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:27.542807 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:27.542807 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:27.542807 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:27.542807 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:27.542807 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:27.542807 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:27.542807 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:27.542807 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:27.542807 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:27.542807 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:27.542807 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:27.543997 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:27.543997 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:27.543997 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:27.543997 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:27.543997 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:27.543997 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:27.543997 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:27.543997 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:27.543997 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:27.543997 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:27.543997 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:27.543997 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:27.543997 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:27.543997 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:27.543997 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:27.543997 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:27.543997 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:27.545159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:27.545159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:27.545159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:27.547087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 409/816] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:27.548532 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8a251c-9189-41c3-be57-07401119e693 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:27.571263 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:27.574685 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c9cabc-61a4-4b11-a3d3-db64d0d90dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:27.575402 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9cabc-61a4-4b11-a3d3-db64d0d90dab None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:27.575795 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9cabc-61a4-4b11-a3d3-db64d0d90dab None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:27.576175 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9cabc-61a4-4b11-a3d3-db64d0d90dab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:27.588982 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c9cabc-61a4-4b11-a3d3-db64d0d90dab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:27.590882 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c9cabc-61a4-4b11-a3d3-db64d0d90dab None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:27.606302 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-e6c9cabc-61a4-4b11-a3d3-db64d0d90dab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:27.606302 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:27.606302 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:27.606302 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:27.606302 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:27.606302 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:27.606302 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:27.606302 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:27.606302 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:27.606302 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:27.606302 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:27.606302 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:27.606302 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:27.606302 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:27.606302 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:27.606302 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:27.607498 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:27.607498 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:27.607498 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:27.607498 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:27.607498 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:27.607498 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:27.607498 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:27.607498 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:27.607498 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:27.607498 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:27.607498 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:27.607498 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:27.607498 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:27.607498 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:27.607498 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:27.607498 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:27.607498 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:27.608636 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:27.608636 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:27.608636 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:27.609917 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 408/817] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:27.610797 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c9cabc-61a4-4b11-a3d3-db64d0d90dab None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:27.687193 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8a251c-9189-41c3-be57-07401119e693 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:27.723791 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65df1b-c415-4b13-8b3d-e96174c2520c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:27.724476 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65df1b-c415-4b13-8b3d-e96174c2520c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:27.724755 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65df1b-c415-4b13-8b3d-e96174c2520c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:27.725027 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65df1b-c415-4b13-8b3d-e96174c2520c None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:27.725839 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b65df1b-c415-4b13-8b3d-e96174c2520c None admin] RBAC: Adding query filter params (name=network) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:27.726466 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b65df1b-c415-4b13-8b3d-e96174c2520c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:27.728329 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b65df1b-c415-4b13-8b3d-e96174c2520c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:27.729169 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:27.733915 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 410/818] 10.222.0.26 () {56 vars in 1112 bytes} [Fri Sep 9 20:15:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:27.734666 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65df1b-c415-4b13-8b3d-e96174c2520c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:27.735565 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c9cabc-61a4-4b11-a3d3-db64d0d90dab None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:27.772840 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308751b6-3a7e-4329-bdc5-293d231bfced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:27.773902 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308751b6-3a7e-4329-bdc5-293d231bfced None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:27.774381 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308751b6-3a7e-4329-bdc5-293d231bfced None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:27.774925 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308751b6-3a7e-4329-bdc5-293d231bfced None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:27.775779 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308751b6-3a7e-4329-bdc5-293d231bfced None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:27.776290 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308751b6-3a7e-4329-bdc5-293d231bfced None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:27.777986 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308751b6-3a7e-4329-bdc5-293d231bfced None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:27.791007 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 409/819] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:27.792155 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308751b6-3a7e-4329-bdc5-293d231bfced None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:27.834193 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:27.863509 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65df1b-c415-4b13-8b3d-e96174c2520c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:27.907205 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308751b6-3a7e-4329-bdc5-293d231bfced None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:27.914161 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a3729-b527-4f9a-a157-85b99530fe89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:27.915075 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a3729-b527-4f9a-a157-85b99530fe89 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:27.915556 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a3729-b527-4f9a-a157-85b99530fe89 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:27.916551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a3729-b527-4f9a-a157-85b99530fe89 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:27.917434 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a3729-b527-4f9a-a157-85b99530fe89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:27.918395 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a3729-b527-4f9a-a157-85b99530fe89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:27.920507 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a3729-b527-4f9a-a157-85b99530fe89 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:27.941784 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 411/820] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:27.943217 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a3729-b527-4f9a-a157-85b99530fe89 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:27.949440 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f167f-1d37-42b7-9b79-1af2efd1f227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:27.950139 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f167f-1d37-42b7-9b79-1af2efd1f227 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:27.950486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f167f-1d37-42b7-9b79-1af2efd1f227 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:27.950822 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f167f-1d37-42b7-9b79-1af2efd1f227 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:27.951607 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69f167f-1d37-42b7-9b79-1af2efd1f227 None admin] RBAC: Adding query filter params (type=network) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:27.951982 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69f167f-1d37-42b7-9b79-1af2efd1f227 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:27.953415 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69f167f-1d37-42b7-9b79-1af2efd1f227 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:27.959042 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 410/821] 10.222.0.26 () {56 vars in 1112 bytes} [Fri Sep 9 20:15:27 2022] GET /identity/v3/services?type=network => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:27.959891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f167f-1d37-42b7-9b79-1af2efd1f227 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:27.987335 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:28.067653 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a3729-b527-4f9a-a157-85b99530fe89 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:28.082506 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f167f-1d37-42b7-9b79-1af2efd1f227 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:28.103931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267f6fc4-ca96-4b21-83f6-ca7eeab275d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:28.104668 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f6fc4-ca96-4b21-83f6-ca7eeab275d5 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:28.105006 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f6fc4-ca96-4b21-83f6-ca7eeab275d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:28.105449 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f6fc4-ca96-4b21-83f6-ca7eeab275d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:28.112229 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267f6fc4-ca96-4b21-83f6-ca7eeab275d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:28.113807 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267f6fc4-ca96-4b21-83f6-ca7eeab275d5 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:28.116283 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:28.126967 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-267f6fc4-ca96-4b21-83f6-ca7eeab275d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:28.126967 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:28.126967 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:28.126967 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:28.126967 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:28.126967 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:28.126967 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:28.126967 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:28.126967 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:28.126967 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:28.126967 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:28.126967 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:28.126967 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:15:28.126967 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:15:28.126967 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:15:28.127916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:15:28.127916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:28.127916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:28.127916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:28.127916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:28.127916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:28.127916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:28.127916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:28.127916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:28.127916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:28.127916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:28.127916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:28.127916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:28.127916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:28.127916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:28.127916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:28.127916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:28.129095 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:15:28.129095 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:15:28.129095 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:15:28.129095 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:15:28.129095 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:15:28.129095 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:15:28.129095 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:28.129095 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:28.129095 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 412/822] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:15:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:28.130601 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990865d-d9e3-4323-b8f2-8490c331694b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:28.131158 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267f6fc4-ca96-4b21-83f6-ca7eeab275d5 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:28.131828 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990865d-d9e3-4323-b8f2-8490c331694b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:28.132276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990865d-d9e3-4323-b8f2-8490c331694b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:28.132812 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990865d-d9e3-4323-b8f2-8490c331694b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:28.142936 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3990865d-d9e3-4323-b8f2-8490c331694b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:28.145289 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3990865d-d9e3-4323-b8f2-8490c331694b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:28.160251 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-3990865d-d9e3-4323-b8f2-8490c331694b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 20:15:28.160251 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:28.160251 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:28.160251 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:28.160251 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:28.160251 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:28.160251 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:28.160251 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:28.160251 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:28.160251 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:28.160251 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:28.160251 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:28.160251 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:15:28.160251 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:15:28.160251 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:15:28.160251 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:15:28.161276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:28.161276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:28.161276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:15:28.161276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:28.161276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:15:28.161276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:28.161276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:28.161276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:28.161276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:28.161276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:28.161276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:28.161276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:28.161276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:28.161276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:28.161276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:28.161276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:28.161276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:28.161276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:28.162807 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:28.162807 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:28.162807 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:15:28.162807 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:15:28.162807 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:15:28.162807 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:15:28.162807 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:15:28.162807 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:15:28.162807 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 20:15:28.162807 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:28.163384 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 411/823] 10.222.0.26 () {56 vars in 1117 bytes} [Fri Sep 9 20:15:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:28.168870 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990865d-d9e3-4323-b8f2-8490c331694b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:28.244217 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267f6fc4-ca96-4b21-83f6-ca7eeab275d5 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:28.276448 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:28.286579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c331696-e81a-42da-bafa-68f339cd453c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:28.288003 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c331696-e81a-42da-bafa-68f339cd453c None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:28.288608 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c331696-e81a-42da-bafa-68f339cd453c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:28.289451 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c331696-e81a-42da-bafa-68f339cd453c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:28.290599 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c331696-e81a-42da-bafa-68f339cd453c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:28.292769 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c331696-e81a-42da-bafa-68f339cd453c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:28.294950 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990865d-d9e3-4323-b8f2-8490c331694b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:28.320518 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 413/824] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:15:28 2022] POST /identity/v3/endpoints => generated 337 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:15:28.322371 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c331696-e81a-42da-bafa-68f339cd453c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:28.362138 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab89af5-7dd9-4434-a4e2-b56d2d3952e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:28.362780 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab89af5-7dd9-4434-a4e2-b56d2d3952e7 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:28.363058 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab89af5-7dd9-4434-a4e2-b56d2d3952e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:28.363329 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab89af5-7dd9-4434-a4e2-b56d2d3952e7 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:28.364044 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab89af5-7dd9-4434-a4e2-b56d2d3952e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:28.364348 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab89af5-7dd9-4434-a4e2-b56d2d3952e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:28.366484 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab89af5-7dd9-4434-a4e2-b56d2d3952e7 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:28.392200 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 412/825] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:15:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:28.400229 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab89af5-7dd9-4434-a4e2-b56d2d3952e7 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:28.450654 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:28.523421 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c331696-e81a-42da-bafa-68f339cd453c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:28.562111 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a5839-048f-44c5-8052-108eba610d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:28.562722 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a5839-048f-44c5-8052-108eba610d87 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:28.563003 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a5839-048f-44c5-8052-108eba610d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:28.563271 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a5839-048f-44c5-8052-108eba610d87 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:28.564087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7a5839-048f-44c5-8052-108eba610d87 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:28.564393 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7a5839-048f-44c5-8052-108eba610d87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:28.565858 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7a5839-048f-44c5-8052-108eba610d87 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:28.585173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 414/826] 10.222.0.26 () {56 vars in 1149 bytes} [Fri Sep 9 20:15:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:28.586856 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e93b09-9a5f-440e-b9c7-c4b6cb440fb4 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:28.587347 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e93b09-9a5f-440e-b9c7-c4b6cb440fb4 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:28.587811 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e93b09-9a5f-440e-b9c7-c4b6cb440fb4 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:28.589201 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 415/827] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:28.591744 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e93b09-9a5f-440e-b9c7-c4b6cb440fb4 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:28.603740 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab89af5-7dd9-4434-a4e2-b56d2d3952e7 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:28.655270 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08262ed2-8069-4f6a-b545-dbbf2f1ff002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:28.655906 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08262ed2-8069-4f6a-b545-dbbf2f1ff002 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:28.656167 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08262ed2-8069-4f6a-b545-dbbf2f1ff002 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:28.656439 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08262ed2-8069-4f6a-b545-dbbf2f1ff002 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:28.657273 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08262ed2-8069-4f6a-b545-dbbf2f1ff002 None admin] RBAC: Adding query filter params (role.id=9121ba25a6d84756990f0c3a19e9fecb, scope.project.id=933820e2adbc473bbd57a4d59be75d72, user.id=87ece6128f9245f7b49f3e7a9faaed22) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:28.657502 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08262ed2-8069-4f6a-b545-dbbf2f1ff002 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:28.658947 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08262ed2-8069-4f6a-b545-dbbf2f1ff002 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:28.666626 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 413/828] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:15:28 2022] GET /identity/v3/role_assignments?user.id=87ece6128f9245f7b49f3e7a9faaed22&scope.project.id=933820e2adbc473bbd57a4d59be75d72&role.id=9121ba25a6d84756990f0c3a19e9fecb => generated 608 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:28.667658 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6543818-dcbf-4345-842f-ba907a745939 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:28.668187 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6543818-dcbf-4345-842f-ba907a745939 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:28.668187 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6543818-dcbf-4345-842f-ba907a745939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:28.678194 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:28.687695 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-c6543818-dcbf-4345-842f-ba907a745939 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:28.688915 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c6543818-dcbf-4345-842f-ba907a745939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:28.730654 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e93b09-9a5f-440e-b9c7-c4b6cb440fb4 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:28.746674 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:28.775142 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9131b0d-0645-4a46-9e6e-7188b040da10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:28.776197 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9131b0d-0645-4a46-9e6e-7188b040da10 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:28.776735 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9131b0d-0645-4a46-9e6e-7188b040da10 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:28.777184 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9131b0d-0645-4a46-9e6e-7188b040da10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:28.788602 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 414/829] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:15:28.793071 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9131b0d-0645-4a46-9e6e-7188b040da10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:28.794580 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9131b0d-0645-4a46-9e6e-7188b040da10 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:28.812251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-a9131b0d-0645-4a46-9e6e-7188b040da10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:28.812251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:28.812251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:28.812251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:28.812251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:28.812251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:28.812251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:28.812251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:28.812251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:28.812251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:28.812251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:28.812251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:28.812251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:28.812251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:28.812251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:28.812251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:28.813527 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:28.813527 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:28.813527 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:28.813527 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:28.813527 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:28.813527 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:28.813527 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:28.813527 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:28.813527 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:28.813527 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:28.813527 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:28.813527 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:28.813527 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:28.813527 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:28.813527 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:28.813527 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:28.813527 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:28.814558 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:28.814558 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:28.814558 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:28.814558 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 416/830] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:28.820047 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6543818-dcbf-4345-842f-ba907a745939 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:28.929209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6543818-dcbf-4345-842f-ba907a745939 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:28.964463 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40836102-3cb6-4a9d-ae26-728a4ce1fae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:28.965083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40836102-3cb6-4a9d-ae26-728a4ce1fae3 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:28.965385 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40836102-3cb6-4a9d-ae26-728a4ce1fae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:28.965631 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40836102-3cb6-4a9d-ae26-728a4ce1fae3 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:28.966268 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40836102-3cb6-4a9d-ae26-728a4ce1fae3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:28.966567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40836102-3cb6-4a9d-ae26-728a4ce1fae3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:28.967917 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40836102-3cb6-4a9d-ae26-728a4ce1fae3 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:28.969435 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:28.979843 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 415/831] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:28.985577 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9131b0d-0645-4a46-9e6e-7188b040da10 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:29.086256 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40836102-3cb6-4a9d-ae26-728a4ce1fae3 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:29.097749 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9131b0d-0645-4a46-9e6e-7188b040da10 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:29.143198 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c855b9b-66c4-4abf-b102-1f6782035643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:29.143855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c855b9b-66c4-4abf-b102-1f6782035643 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:29.144140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c855b9b-66c4-4abf-b102-1f6782035643 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:29.144417 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c855b9b-66c4-4abf-b102-1f6782035643 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:29.154049 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c855b9b-66c4-4abf-b102-1f6782035643 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:29.155835 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c855b9b-66c4-4abf-b102-1f6782035643 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:29.174446 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-5c855b9b-66c4-4abf-b102-1f6782035643 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:29.174446 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:29.174446 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:29.174446 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:29.174446 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:29.174446 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:29.174446 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:29.174446 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:29.174446 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:29.174446 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:29.174446 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:29.174446 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:29.174446 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:15:29.174446 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:15:29.174446 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:15:29.175443 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:15:29.175443 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:29.175443 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:29.175443 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:29.175443 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:29.175443 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:29.175443 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:29.175443 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:29.175443 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:29.175443 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:29.175443 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:29.175443 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:29.175443 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:29.175443 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:29.175443 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:29.175443 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:29.175443 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:29.176385 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:15:29.176385 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:15:29.176385 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:15:29.176385 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:15:29.176385 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:15:29.176385 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:15:29.176385 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:29.176385 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:29.176385 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 417/832] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:15:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:29.183070 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c855b9b-66c4-4abf-b102-1f6782035643 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:29.214238 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40836102-3cb6-4a9d-ae26-728a4ce1fae3 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:29.257105 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e8112-cced-4bb8-b075-8d4f5ec38dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:29.258184 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e8112-cced-4bb8-b075-8d4f5ec38dd5 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:29.258702 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e8112-cced-4bb8-b075-8d4f5ec38dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:29.259180 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e8112-cced-4bb8-b075-8d4f5ec38dd5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:29.272191 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574e8112-cced-4bb8-b075-8d4f5ec38dd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:15:29.278133 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-574e8112-cced-4bb8-b075-8d4f5ec38dd5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:15:29.278133 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:29.278133 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:29.278133 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:29.278133 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:29.278133 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:29.278133 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:29.278133 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:29.278133 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:29.278133 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:29.278133 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:29.278133 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:29.278133 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:15:29.278133 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:15:29.278133 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:15:29.278133 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:15:29.279171 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:15:29.279171 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:15:29.279171 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:15:29.279171 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:15:29.279171 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:29.279171 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:29.279171 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:29.279171 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:29.279171 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:29.279171 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:29.279171 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:29.279171 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:29.279171 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:29.279171 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:29.279171 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:29.279171 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:29.280055 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:29.280055 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:29.280055 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:29.280055 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:29.280055 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:15:29.280055 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:15:29.280055 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:15:29.280055 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:15:29.280055 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:15:29.280055 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:15:29.280055 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:15:29.280055 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:29.283130 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 416/833] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:15:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:15:29.284706 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68169fb5-52da-4119-b15b-56cb42ae10be None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:29.285210 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68169fb5-52da-4119-b15b-56cb42ae10be None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:29.285739 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68169fb5-52da-4119-b15b-56cb42ae10be None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:29.287696 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 417/834] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:29.288726 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68169fb5-52da-4119-b15b-56cb42ae10be None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:29.340871 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:29.350730 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:29.403098 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68169fb5-52da-4119-b15b-56cb42ae10be None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:29.423248 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c855b9b-66c4-4abf-b102-1f6782035643 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:29.440066 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239a25a-db8a-47c9-9fb3-f05ba988dbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:29.440757 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239a25a-db8a-47c9-9fb3-f05ba988dbc7 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:29.441047 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239a25a-db8a-47c9-9fb3-f05ba988dbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:29.441345 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239a25a-db8a-47c9-9fb3-f05ba988dbc7 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:29.442014 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2239a25a-db8a-47c9-9fb3-f05ba988dbc7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:29.442409 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2239a25a-db8a-47c9-9fb3-f05ba988dbc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:29.443700 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2239a25a-db8a-47c9-9fb3-f05ba988dbc7 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:29.452096 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 418/835] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:15:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:29.453352 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb4a29-f120-4609-b9f8-4497ff49f175 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:29.453639 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb4a29-f120-4609-b9f8-4497ff49f175 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:29.453913 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb4a29-f120-4609-b9f8-4497ff49f175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:29.465211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3b2a8-dad5-417e-b128-b6e6ce805db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:29.465914 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3b2a8-dad5-417e-b128-b6e6ce805db5 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:29.466254 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3b2a8-dad5-417e-b128-b6e6ce805db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:29.466586 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3b2a8-dad5-417e-b128-b6e6ce805db5 None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:29.467363 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb3b2a8-dad5-417e-b128-b6e6ce805db5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:29.467695 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb3b2a8-dad5-417e-b128-b6e6ce805db5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:29.469186 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb3b2a8-dad5-417e-b128-b6e6ce805db5 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:29.475450 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-4eeb4a29-f120-4609-b9f8-4497ff49f175 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:29.477473 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4eeb4a29-f120-4609-b9f8-4497ff49f175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:29.486586 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 418/836] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:15:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:29.486586 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3b2a8-dad5-417e-b128-b6e6ce805db5 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:29.557834 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 419/837] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:29 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:15:29.557834 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb4a29-f120-4609-b9f8-4497ff49f175 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:29.575788 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:29.600438 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3b2a8-dad5-417e-b128-b6e6ce805db5 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:29.609872 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:29.652512 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41516981-f606-47b2-8f54-72a9438d858a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:29.653966 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41516981-f606-47b2-8f54-72a9438d858a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:29.654443 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41516981-f606-47b2-8f54-72a9438d858a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:29.654866 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41516981-f606-47b2-8f54-72a9438d858a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:29.678605 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41516981-f606-47b2-8f54-72a9438d858a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:29.681933 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb4a29-f120-4609-b9f8-4497ff49f175 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:29.682321 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41516981-f606-47b2-8f54-72a9438d858a None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:29.699778 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-41516981-f606-47b2-8f54-72a9438d858a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:29.699778 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:29.699778 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:29.699778 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:29.699778 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:29.699778 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:29.699778 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:29.699778 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:29.699778 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:29.699778 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:29.699778 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:29.699778 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:29.699778 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:29.699778 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:29.699778 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:29.699778 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:29.700998 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:29.700998 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:29.700998 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:29.700998 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:29.700998 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:29.700998 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:29.700998 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:29.700998 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:29.700998 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:29.700998 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:29.700998 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:29.700998 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:29.700998 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:29.700998 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:29.700998 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:29.700998 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:29.700998 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:29.702176 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:29.702176 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:29.702176 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:29.707864 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 419/838] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:29.709362 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41516981-f606-47b2-8f54-72a9438d858a None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:29.733899 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543c1fa-7ee1-4f9d-99df-8c91f0da34c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:29.734370 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543c1fa-7ee1-4f9d-99df-8c91f0da34c8 None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:29.734648 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543c1fa-7ee1-4f9d-99df-8c91f0da34c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:29.734918 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543c1fa-7ee1-4f9d-99df-8c91f0da34c8 None admin] PATH_INFO: `/v3/projects/933820e2adbc473bbd57a4d59be75d72/users/cedaf156c5584d62beedde598c94f322/roles/8ec32f702e0c447993ff294cbac536b1` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:29.753667 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e543c1fa-7ee1-4f9d-99df-8c91f0da34c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=933820e2adbc473bbd57a4d59be75d72, user_id=cedaf156c5584d62beedde598c94f322, role_id=8ec32f702e0c447993ff294cbac536b1)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:29.755368 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e543c1fa-7ee1-4f9d-99df-8c91f0da34c8 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:29.771663 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 420/839] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Sep 9 20:15:29 2022] PUT /identity/v3/projects/933820e2adbc473bbd57a4d59be75d72/users/cedaf156c5584d62beedde598c94f322/roles/8ec32f702e0c447993ff294cbac536b1 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:15:29.832709 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41516981-f606-47b2-8f54-72a9438d858a None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:29.843875 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543c1fa-7ee1-4f9d-99df-8c91f0da34c8 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:29.881175 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2df22-5505-4b6d-b08f-6b7f72c8a8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:29.881785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2df22-5505-4b6d-b08f-6b7f72c8a8c7 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:29.882063 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2df22-5505-4b6d-b08f-6b7f72c8a8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:29.882360 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2df22-5505-4b6d-b08f-6b7f72c8a8c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:29.883076 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2df22-5505-4b6d-b08f-6b7f72c8a8c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:29.883470 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2df22-5505-4b6d-b08f-6b7f72c8a8c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:29.885000 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2df22-5505-4b6d-b08f-6b7f72c8a8c7 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:29.887922 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:29.901159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 420/840] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:29.908354 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2df22-5505-4b6d-b08f-6b7f72c8a8c7 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:29.977301 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:29.985960 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543c1fa-7ee1-4f9d-99df-8c91f0da34c8 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:30.028135 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3dede8-cf44-436d-b8bf-7d1f2a84f29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:30.028778 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3dede8-cf44-436d-b8bf-7d1f2a84f29c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:30.029124 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3dede8-cf44-436d-b8bf-7d1f2a84f29c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:30.029515 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3dede8-cf44-436d-b8bf-7d1f2a84f29c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:30.030177 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3dede8-cf44-436d-b8bf-7d1f2a84f29c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:30.031864 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3dede8-cf44-436d-b8bf-7d1f2a84f29c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:30.043238 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2df22-5505-4b6d-b08f-6b7f72c8a8c7 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:30.044475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-7f3dede8-cf44-436d-b8bf-7d1f2a84f29c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 20:15:30.044475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:30.044475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:15:30.044475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:15:30.044475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:15:30.044475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:15:30.044475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:15:30.044475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:30.044475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 20:15:30.044475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:30.044475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:15:30.044475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:30.044475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:30.044475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:30.044475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:30.044475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:30.044475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:30.045475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:30.045475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:30.045475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:30.045475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:30.045475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:30.045475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:30.045475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:15:30.045475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:15:30.045475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:15:30.045475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:15:30.045475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:30.045475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:30.045475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:30.045475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:30.045475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:30.045475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:30.046681 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:30.046681 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:30.046681 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:30.046681 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:30.046681 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:30.046681 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:30.046681 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:30.046681 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:30.046681 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:30.046681 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:30.046681 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:15:30.046681 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:30.046681 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 20:15:30.046681 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:30.046681 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 421/841] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Sep 9 20:15:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:15:30.054342 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3dede8-cf44-436d-b8bf-7d1f2a84f29c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:30.086583 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44019091-7b20-42f6-afcb-2ecc48dce863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:30.087312 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44019091-7b20-42f6-afcb-2ecc48dce863 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:30.087720 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44019091-7b20-42f6-afcb-2ecc48dce863 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:30.088266 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44019091-7b20-42f6-afcb-2ecc48dce863 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:30.098511 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44019091-7b20-42f6-afcb-2ecc48dce863 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:30.101199 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44019091-7b20-42f6-afcb-2ecc48dce863 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:30.115847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-44019091-7b20-42f6-afcb-2ecc48dce863 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 20:15:30.115847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:30.115847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:30.115847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:30.115847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:30.115847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:30.115847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:30.115847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:30.115847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:30.115847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:30.115847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:30.115847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:30.115847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:15:30.115847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:15:30.115847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:15:30.115847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:15:30.117055 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:30.117055 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:30.117055 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:15:30.117055 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:30.117055 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:15:30.117055 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:30.117055 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:30.117055 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:30.117055 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:30.117055 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:30.117055 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:30.117055 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:30.117055 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:30.117055 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:30.117055 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:30.117055 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:30.117055 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:30.117055 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:30.118324 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:30.118324 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:30.118324 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:15:30.118324 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:15:30.118324 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:15:30.118324 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:15:30.118324 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:15:30.118324 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:15:30.118324 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 20:15:30.118324 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:30.119451 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 421/842] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:15:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:15:30.124837 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44019091-7b20-42f6-afcb-2ecc48dce863 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:30.185629 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:30.197684 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3dede8-cf44-436d-b8bf-7d1f2a84f29c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:30.234617 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60046c20-6f8a-4c2d-a4b7-b35e25044165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:30.235315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60046c20-6f8a-4c2d-a4b7-b35e25044165 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:30.235546 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60046c20-6f8a-4c2d-a4b7-b35e25044165 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:30.235767 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60046c20-6f8a-4c2d-a4b7-b35e25044165 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:30.236433 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60046c20-6f8a-4c2d-a4b7-b35e25044165 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:30.236736 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60046c20-6f8a-4c2d-a4b7-b35e25044165 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:30.238093 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60046c20-6f8a-4c2d-a4b7-b35e25044165 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:30.247827 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 422/843] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Sep 9 20:15:30 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:30.250220 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60046c20-6f8a-4c2d-a4b7-b35e25044165 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:30.252127 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44019091-7b20-42f6-afcb-2ecc48dce863 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:30.284527 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:30.288518 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c85fb-8302-44a9-9f4f-469e3ff7e755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:30.289208 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c85fb-8302-44a9-9f4f-469e3ff7e755 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:30.289989 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c85fb-8302-44a9-9f4f-469e3ff7e755 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:30.290614 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c85fb-8302-44a9-9f4f-469e3ff7e755 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:30.291925 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c85fb-8302-44a9-9f4f-469e3ff7e755 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:30.292323 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c85fb-8302-44a9-9f4f-469e3ff7e755 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:30.294040 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c85fb-8302-44a9-9f4f-469e3ff7e755 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:30.312929 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 422/844] 10.222.0.26 () {56 vars in 1135 bytes} [Fri Sep 9 20:15:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:30.318105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c85fb-8302-44a9-9f4f-469e3ff7e755 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:30.386785 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60046c20-6f8a-4c2d-a4b7-b35e25044165 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:30.426992 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb7ac9-24d8-44dd-b481-034317621cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:30.428155 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb7ac9-24d8-44dd-b481-034317621cef None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:30.428437 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb7ac9-24d8-44dd-b481-034317621cef None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:30.428710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb7ac9-24d8-44dd-b481-034317621cef None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:30.429529 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb7ac9-24d8-44dd-b481-034317621cef None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:30.429835 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb7ac9-24d8-44dd-b481-034317621cef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:30.431652 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb7ac9-24d8-44dd-b481-034317621cef None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:30.443538 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 423/845] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Sep 9 20:15:30 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:30.452597 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c85fb-8302-44a9-9f4f-469e3ff7e755 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:30.470093 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:30.494829 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672cfeb-2dc2-4038-9135-bba49441a673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:30.495830 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672cfeb-2dc2-4038-9135-bba49441a673 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:30.496315 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672cfeb-2dc2-4038-9135-bba49441a673 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:30.496811 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672cfeb-2dc2-4038-9135-bba49441a673 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:30.579134 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672cfeb-2dc2-4038-9135-bba49441a673 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:30.580615 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672cfeb-2dc2-4038-9135-bba49441a673 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:30.702884 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 423/846] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:15:30 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 20:15:30.709280 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb7ac9-24d8-44dd-b481-034317621cef None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:30.809273 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:30.823556 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb7ac9-24d8-44dd-b481-034317621cef None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:30.863070 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097cf3aa-9268-48b8-90c8-ec03de3c0cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:30.864491 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cf3aa-9268-48b8-90c8-ec03de3c0cf2 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:30.865065 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cf3aa-9268-48b8-90c8-ec03de3c0cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:30.865631 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cf3aa-9268-48b8-90c8-ec03de3c0cf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:30.876482 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097cf3aa-9268-48b8-90c8-ec03de3c0cf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:30.878263 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097cf3aa-9268-48b8-90c8-ec03de3c0cf2 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:30.896576 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-097cf3aa-9268-48b8-90c8-ec03de3c0cf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:30.896576 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:30.896576 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:30.896576 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:30.896576 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:30.896576 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:30.896576 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:30.896576 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:30.896576 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:30.896576 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:30.896576 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:30.896576 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:30.896576 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:30.896576 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:30.896576 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:30.896576 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:30.897672 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:30.897672 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:30.897672 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:30.897672 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:30.897672 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:30.897672 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:30.897672 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:30.897672 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:30.897672 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:30.897672 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:30.897672 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:30.897672 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:30.897672 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:30.897672 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:30.897672 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:30.897672 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:30.897672 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:30.898597 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:30.898597 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:30.898597 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:30.900895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 424/847] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:30.910441 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672cfeb-2dc2-4038-9135-bba49441a673 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:30.926911 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:31.010783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5ee3d-1027-4a0e-a1b7-76527a125257 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:31.011130 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5ee3d-1027-4a0e-a1b7-76527a125257 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:31.011375 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5ee3d-1027-4a0e-a1b7-76527a125257 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:31.013003 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 425/848] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:31.017989 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743c29f-2627-433e-a8b0-093f312567bf None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:31.018277 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743c29f-2627-433e-a8b0-093f312567bf None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:31.018612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743c29f-2627-433e-a8b0-093f312567bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:31.022602 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672cfeb-2dc2-4038-9135-bba49441a673 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:31.040056 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-5743c29f-2627-433e-a8b0-093f312567bf None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:31.041313 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5743c29f-2627-433e-a8b0-093f312567bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:31.070652 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36bccc8-ed36-4c38-b172-f32c5a1cd627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:31.072589 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36bccc8-ed36-4c38-b172-f32c5a1cd627 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:31.073094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36bccc8-ed36-4c38-b172-f32c5a1cd627 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:31.073760 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36bccc8-ed36-4c38-b172-f32c5a1cd627 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:31.074934 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36bccc8-ed36-4c38-b172-f32c5a1cd627 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:31.075687 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36bccc8-ed36-4c38-b172-f32c5a1cd627 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:31.077386 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36bccc8-ed36-4c38-b172-f32c5a1cd627 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:31.092727 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 424/849] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:31.101292 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36bccc8-ed36-4c38-b172-f32c5a1cd627 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:31.119138 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 426/850] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:31 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:15:31.205696 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:31.214864 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36bccc8-ed36-4c38-b172-f32c5a1cd627 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:31.249101 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ae49d-cd0b-4375-82f4-fcf55bf1729a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:31.249821 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ae49d-cd0b-4375-82f4-fcf55bf1729a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:31.250098 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ae49d-cd0b-4375-82f4-fcf55bf1729a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:31.250371 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ae49d-cd0b-4375-82f4-fcf55bf1729a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:31.258153 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6ae49d-cd0b-4375-82f4-fcf55bf1729a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:31.259587 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6ae49d-cd0b-4375-82f4-fcf55bf1729a None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:31.271655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-ea6ae49d-cd0b-4375-82f4-fcf55bf1729a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:31.271655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:31.271655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:31.271655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:31.271655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:31.271655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:31.271655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:31.271655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:31.271655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:31.271655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:31.271655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:31.271655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:31.271655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:15:31.271655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:15:31.271655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:15:31.272534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:15:31.272534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:31.272534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:31.272534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:31.272534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:31.272534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:31.272534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:31.272534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:31.272534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:31.272534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:31.272534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:31.272534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:31.272534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:31.272534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:31.272534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:31.272534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:31.272534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:31.273489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:15:31.273489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:15:31.273489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:15:31.273489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:15:31.273489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:15:31.273489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:15:31.273489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:31.273489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:31.273489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 425/851] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:15:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:31.279114 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743c29f-2627-433e-a8b0-093f312567bf None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:31.351574 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:31.384800 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743c29f-2627-433e-a8b0-093f312567bf None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:31.400463 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ae49d-cd0b-4375-82f4-fcf55bf1729a None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:31.425664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a7200-08d3-4677-90dd-8ac65a87dce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:31.426605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a7200-08d3-4677-90dd-8ac65a87dce9 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:31.426893 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a7200-08d3-4677-90dd-8ac65a87dce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:31.427740 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a7200-08d3-4677-90dd-8ac65a87dce9 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:31.428619 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a7200-08d3-4677-90dd-8ac65a87dce9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:31.428922 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a7200-08d3-4677-90dd-8ac65a87dce9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:31.430992 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a7200-08d3-4677-90dd-8ac65a87dce9 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:31.447101 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 427/852] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:15:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:31.453942 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a7200-08d3-4677-90dd-8ac65a87dce9 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:31.534994 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ae49d-cd0b-4375-82f4-fcf55bf1729a None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:31.560673 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a7200-08d3-4677-90dd-8ac65a87dce9 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:31.571043 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc98732-e5af-4a5a-9cd2-003a62c18742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:31.571930 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc98732-e5af-4a5a-9cd2-003a62c18742 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:31.572417 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc98732-e5af-4a5a-9cd2-003a62c18742 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:31.572877 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc98732-e5af-4a5a-9cd2-003a62c18742 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:31.573720 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc98732-e5af-4a5a-9cd2-003a62c18742 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:31.575269 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc98732-e5af-4a5a-9cd2-003a62c18742 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:31.577172 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:31.586374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-0cc98732-e5af-4a5a-9cd2-003a62c18742 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 20:15:31.586374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:31.586374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:15:31.586374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:15:31.586374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:15:31.586374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:15:31.586374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:15:31.586374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:31.586374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 20:15:31.586374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:31.586374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:15:31.586374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:31.586374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:31.586374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:31.586374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:31.586374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:31.587326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:31.587326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:31.587326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:31.587326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:31.587326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:31.587326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:31.587326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:31.587326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:15:31.587326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:15:31.587326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:15:31.587326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:15:31.587326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:31.587326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:31.587326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:31.587326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:31.587326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:31.587326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:31.588213 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:31.588213 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:31.588213 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:31.588213 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:31.588213 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:31.588213 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:31.588213 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:31.588213 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:31.588213 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:31.588213 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:31.588213 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:15:31.588213 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:31.588213 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 20:15:31.588213 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:31.590910 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 426/853] 10.222.0.26 () {56 vars in 1132 bytes} [Fri Sep 9 20:15:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:15:31.593049 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:31.598139 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc98732-e5af-4a5a-9cd2-003a62c18742 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:31.602066 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68ea6e-1d8b-4685-a2b8-9c32e7b837b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:31.602679 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68ea6e-1d8b-4685-a2b8-9c32e7b837b4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:31.602957 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68ea6e-1d8b-4685-a2b8-9c32e7b837b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:31.603233 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68ea6e-1d8b-4685-a2b8-9c32e7b837b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:31.603960 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68ea6e-1d8b-4685-a2b8-9c32e7b837b4 None admin] RBAC: Adding query filter params (role.id=9121ba25a6d84756990f0c3a19e9fecb, scope.project.id=933820e2adbc473bbd57a4d59be75d72, user.id=fcb62d06f7e94e9295c54504ddabb1d4) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:31.604274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68ea6e-1d8b-4685-a2b8-9c32e7b837b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:31.605726 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68ea6e-1d8b-4685-a2b8-9c32e7b837b4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:31.616889 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 428/854] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:15:31 2022] GET /identity/v3/role_assignments?user.id=fcb62d06f7e94e9295c54504ddabb1d4&scope.project.id=933820e2adbc473bbd57a4d59be75d72&role.id=9121ba25a6d84756990f0c3a19e9fecb => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:31.715410 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc98732-e5af-4a5a-9cd2-003a62c18742 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:31.752923 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198912af-e56f-413a-86b2-eb004f59969b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:31.753572 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198912af-e56f-413a-86b2-eb004f59969b None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:31.754110 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198912af-e56f-413a-86b2-eb004f59969b None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:31.754379 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198912af-e56f-413a-86b2-eb004f59969b None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:31.755009 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198912af-e56f-413a-86b2-eb004f59969b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:31.755906 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198912af-e56f-413a-86b2-eb004f59969b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:31.757347 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198912af-e56f-413a-86b2-eb004f59969b None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:31.763015 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 427/855] 10.222.0.26 () {56 vars in 1126 bytes} [Fri Sep 9 20:15:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:31.768825 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68ea6e-1d8b-4685-a2b8-9c32e7b837b4 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:31.885340 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68ea6e-1d8b-4685-a2b8-9c32e7b837b4 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:31.920729 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ea43b-33dd-482e-ac9d-69ef374c0f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:31.921784 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ea43b-33dd-482e-ac9d-69ef374c0f3e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:31.922305 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ea43b-33dd-482e-ac9d-69ef374c0f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:31.922772 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ea43b-33dd-482e-ac9d-69ef374c0f3e None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:31.923592 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ea43b-33dd-482e-ac9d-69ef374c0f3e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:31.924087 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ea43b-33dd-482e-ac9d-69ef374c0f3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:31.925682 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ea43b-33dd-482e-ac9d-69ef374c0f3e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:31.931574 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 429/856] 10.222.0.26 () {56 vars in 1126 bytes} [Fri Sep 9 20:15:31 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:32.071684 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568c296-c0a4-436e-b79e-2ffbd946482f None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:32.072264 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568c296-c0a4-436e-b79e-2ffbd946482f None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:32.072264 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568c296-c0a4-436e-b79e-2ffbd946482f None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:32.073973 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 428/857] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:32.079892 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915145a8-8dcd-4221-877e-2af43732c9b7 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:32.080182 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915145a8-8dcd-4221-877e-2af43732c9b7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:32.080459 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915145a8-8dcd-4221-877e-2af43732c9b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:32.100634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-915145a8-8dcd-4221-877e-2af43732c9b7 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:32.101875 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-915145a8-8dcd-4221-877e-2af43732c9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:32.157806 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:32.173543 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 430/858] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:32 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:15:32.453727 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568c296-c0a4-436e-b79e-2ffbd946482f None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:32.520672 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:32.561922 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568c296-c0a4-436e-b79e-2ffbd946482f None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:32.596760 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76019c03-ed75-4b9c-a6aa-66c0d517b156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:32.597445 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76019c03-ed75-4b9c-a6aa-66c0d517b156 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:32.597717 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76019c03-ed75-4b9c-a6aa-66c0d517b156 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:32.597986 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76019c03-ed75-4b9c-a6aa-66c0d517b156 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:32.610911 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76019c03-ed75-4b9c-a6aa-66c0d517b156 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:15:32.615726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-76019c03-ed75-4b9c-a6aa-66c0d517b156 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:15:32.615726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:32.615726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:32.615726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:32.615726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:32.615726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:32.615726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:32.615726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:32.615726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:32.615726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:32.615726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:32.615726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:32.615726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:15:32.615726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:15:32.615726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:15:32.615726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:15:32.617028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:15:32.617028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:15:32.617028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:15:32.617028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:15:32.617028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:32.617028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:32.617028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:32.617028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:32.617028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:32.617028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:32.617028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:32.617028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:32.617028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:32.617028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:32.617028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:32.617028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:32.618274 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:32.618274 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:32.618274 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:32.618274 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:32.618274 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:15:32.618274 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:15:32.618274 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:15:32.618274 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:15:32.618274 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:15:32.618274 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:15:32.618274 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:15:32.618274 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:32.618274 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 429/859] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Sep 9 20:15:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:32.624049 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915145a8-8dcd-4221-877e-2af43732c9b7 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:32.735724 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915145a8-8dcd-4221-877e-2af43732c9b7 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:32.772297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e6520-6139-43c2-9f46-04359fa3c0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:32.772907 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e6520-6139-43c2-9f46-04359fa3c0c3 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:32.773192 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e6520-6139-43c2-9f46-04359fa3c0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:32.773496 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e6520-6139-43c2-9f46-04359fa3c0c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:32.774165 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e6520-6139-43c2-9f46-04359fa3c0c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:32.774472 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e6520-6139-43c2-9f46-04359fa3c0c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:32.775830 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e6520-6139-43c2-9f46-04359fa3c0c3 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:32.784836 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 431/860] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Sep 9 20:15:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:32.791451 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76019c03-ed75-4b9c-a6aa-66c0d517b156 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:32.829202 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966cc21-f311-4a05-bfc5-e95b40a78480 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:32.829631 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966cc21-f311-4a05-bfc5-e95b40a78480 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:32.829942 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966cc21-f311-4a05-bfc5-e95b40a78480 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:32.831919 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 432/861] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:15:32.838004 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5f01b-4593-4390-b450-2130b6163980 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:32.838449 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5f01b-4593-4390-b450-2130b6163980 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:32.838882 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5f01b-4593-4390-b450-2130b6163980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:32.858674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-b7c5f01b-4593-4390-b450-2130b6163980 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:32.860266 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b7c5f01b-4593-4390-b450-2130b6163980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:32.913029 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76019c03-ed75-4b9c-a6aa-66c0d517b156 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:32.920561 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:32.942106 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:32.952409 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 433/862] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:32 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:15:32.963912 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f31984-e006-4a10-9c16-bd9c9fde0775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:32.964597 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f31984-e006-4a10-9c16-bd9c9fde0775 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:32.964879 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f31984-e006-4a10-9c16-bd9c9fde0775 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:32.965206 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f31984-e006-4a10-9c16-bd9c9fde0775 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:32.976616 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f31984-e006-4a10-9c16-bd9c9fde0775 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:32.978086 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f31984-e006-4a10-9c16-bd9c9fde0775 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:32.995773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-d5f31984-e006-4a10-9c16-bd9c9fde0775 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:32.995773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:32.995773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:32.995773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:32.995773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:32.995773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:32.995773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:32.995773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:32.995773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:32.995773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:32.995773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:32.995773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:32.995773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:32.995773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:32.995773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:32.995773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:32.996729 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:32.996729 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:32.996729 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:32.996729 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:32.996729 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:32.996729 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:32.996729 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:32.996729 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:32.996729 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:32.996729 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:32.996729 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:32.996729 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:32.996729 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:32.996729 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:32.996729 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:32.996729 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:32.996729 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:32.997744 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:32.997744 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:32.997744 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:32.997744 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 430/863] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:33.004078 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c5f01b-4593-4390-b450-2130b6163980 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:33.115365 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c5f01b-4593-4390-b450-2130b6163980 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:33.150574 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abffc323-1eac-46b6-b10a-3ed6834fb24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:33.151482 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abffc323-1eac-46b6-b10a-3ed6834fb24a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:33.151982 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abffc323-1eac-46b6-b10a-3ed6834fb24a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:33.152457 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abffc323-1eac-46b6-b10a-3ed6834fb24a None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:33.153307 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abffc323-1eac-46b6-b10a-3ed6834fb24a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:33.153800 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abffc323-1eac-46b6-b10a-3ed6834fb24a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:33.155564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abffc323-1eac-46b6-b10a-3ed6834fb24a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:33.168175 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 434/864] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:33.175658 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f31984-e006-4a10-9c16-bd9c9fde0775 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:33.234588 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abffc323-1eac-46b6-b10a-3ed6834fb24a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:33.289738 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:33.299097 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f31984-e006-4a10-9c16-bd9c9fde0775 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:33.339905 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d1a1b-1232-49c9-9738-246123312b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:33.341060 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d1a1b-1232-49c9-9738-246123312b75 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:33.341704 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d1a1b-1232-49c9-9738-246123312b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:33.342044 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d1a1b-1232-49c9-9738-246123312b75 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:33.353285 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d1a1b-1232-49c9-9738-246123312b75 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:33.355499 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d1a1b-1232-49c9-9738-246123312b75 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:33.360178 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abffc323-1eac-46b6-b10a-3ed6834fb24a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:33.370831 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-749d1a1b-1232-49c9-9738-246123312b75 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 20:15:33.370831 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:33.370831 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:33.370831 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:33.370831 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:33.370831 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:33.370831 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:33.370831 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:33.370831 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:33.370831 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:33.370831 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:33.370831 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:33.370831 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:15:33.370831 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:15:33.370831 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:15:33.370831 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:15:33.372640 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:33.372640 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:33.372640 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:15:33.372640 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:33.372640 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:15:33.372640 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:33.372640 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:33.372640 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:33.372640 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:33.372640 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:33.372640 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:33.372640 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:33.372640 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:33.372640 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:33.372640 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:33.372640 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:33.372640 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:33.372640 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:33.374958 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:33.374958 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:33.374958 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:15:33.374958 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:15:33.374958 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:15:33.374958 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:15:33.374958 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:15:33.374958 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:15:33.374958 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 20:15:33.374958 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:33.381275 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 431/865] 10.222.0.26 () {56 vars in 1117 bytes} [Fri Sep 9 20:15:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:33.381275 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d1a1b-1232-49c9-9738-246123312b75 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:33.405521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954267d-28b6-4a70-baa0-f87f2c870372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:33.406185 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954267d-28b6-4a70-baa0-f87f2c870372 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:33.406475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954267d-28b6-4a70-baa0-f87f2c870372 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:33.406755 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954267d-28b6-4a70-baa0-f87f2c870372 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:33.407462 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d954267d-28b6-4a70-baa0-f87f2c870372 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:33.408883 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d954267d-28b6-4a70-baa0-f87f2c870372 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:33.430590 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:33.430866 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 435/866] 10.222.0.26 () {60 vars in 1141 bytes} [Fri Sep 9 20:15:33 2022] POST /identity/v3/services => generated 251 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:15:33.508343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d1a1b-1232-49c9-9738-246123312b75 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:33.554034 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8fbe77-30ea-4368-b9dd-04d886cfa7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:33.554925 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fbe77-30ea-4368-b9dd-04d886cfa7f8 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:33.555364 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fbe77-30ea-4368-b9dd-04d886cfa7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:33.555795 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fbe77-30ea-4368-b9dd-04d886cfa7f8 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:33.556690 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8fbe77-30ea-4368-b9dd-04d886cfa7f8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:33.557161 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8fbe77-30ea-4368-b9dd-04d886cfa7f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:33.558814 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8fbe77-30ea-4368-b9dd-04d886cfa7f8 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:33.580894 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 432/867] 10.222.0.26 () {56 vars in 1149 bytes} [Fri Sep 9 20:15:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:33.588522 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954267d-28b6-4a70-baa0-f87f2c870372 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:33.711349 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:33.715116 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954267d-28b6-4a70-baa0-f87f2c870372 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:33.752953 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f59b671-d3b0-42f5-8eef-1f61bcee787f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:33.754064 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59b671-d3b0-42f5-8eef-1f61bcee787f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:33.754357 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59b671-d3b0-42f5-8eef-1f61bcee787f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:33.754683 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59b671-d3b0-42f5-8eef-1f61bcee787f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:33.826720 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59b671-d3b0-42f5-8eef-1f61bcee787f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:33.828133 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59b671-d3b0-42f5-8eef-1f61bcee787f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:33.941625 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:33.949616 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 436/868] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:15:33 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:15:33.951734 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8fbe77-30ea-4368-b9dd-04d886cfa7f8 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:33.960305 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3647fae-fd3f-4fab-ba0a-a5b8cd71b27f None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:33.960730 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3647fae-fd3f-4fab-ba0a-a5b8cd71b27f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:33.961068 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3647fae-fd3f-4fab-ba0a-a5b8cd71b27f None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:33.962679 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 437/869] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:15:33.969301 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960fa7de-dc9e-4a90-84de-c6035b8a6b18 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:33.969853 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960fa7de-dc9e-4a90-84de-c6035b8a6b18 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:33.970355 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960fa7de-dc9e-4a90-84de-c6035b8a6b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:33.991267 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-960fa7de-dc9e-4a90-84de-c6035b8a6b18 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:33.992483 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-960fa7de-dc9e-4a90-84de-c6035b8a6b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:34.065956 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:34.081495 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 438/870] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:33 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:15:34.088796 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8fbe77-30ea-4368-b9dd-04d886cfa7f8 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:34.127614 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefbfeba-3288-4401-98bd-3730f6959c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:34.128531 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefbfeba-3288-4401-98bd-3730f6959c90 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:34.129216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefbfeba-3288-4401-98bd-3730f6959c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:34.129669 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefbfeba-3288-4401-98bd-3730f6959c90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:34.142200 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefbfeba-3288-4401-98bd-3730f6959c90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:34.144081 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefbfeba-3288-4401-98bd-3730f6959c90 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:34.160555 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-eefbfeba-3288-4401-98bd-3730f6959c90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:34.160555 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:34.160555 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:34.160555 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:34.160555 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:34.160555 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:34.160555 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:34.160555 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:34.160555 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:34.160555 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:34.160555 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:34.160555 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:34.160555 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:34.160555 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:34.160555 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:34.160555 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:34.162376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:34.162376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:34.162376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:34.162376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:34.162376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:34.162376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:34.162376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:34.162376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:34.162376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:34.162376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:34.162376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:34.162376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:34.162376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:34.162376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:34.162376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:34.162376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:34.162376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:34.163526 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:34.163526 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:34.163526 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:34.169271 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 433/871] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:34.170998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960fa7de-dc9e-4a90-84de-c6035b8a6b18 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:34.239871 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718f8bc-4a4b-426b-a292-a10498fe6263 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:34.240416 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718f8bc-4a4b-426b-a292-a10498fe6263 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:34.240504 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718f8bc-4a4b-426b-a292-a10498fe6263 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:34.242080 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 434/872] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:15:34.250840 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da172402-0d51-45d7-9782-8018f17a8dcc None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:34.251244 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da172402-0d51-45d7-9782-8018f17a8dcc None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:34.251635 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da172402-0d51-45d7-9782-8018f17a8dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:34.276885 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-da172402-0d51-45d7-9782-8018f17a8dcc None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:34.278287 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-da172402-0d51-45d7-9782-8018f17a8dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:34.289799 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960fa7de-dc9e-4a90-84de-c6035b8a6b18 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:34.326971 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:34.335569 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75066e8b-e801-4c52-bfb0-3409ea954787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:34.336123 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75066e8b-e801-4c52-bfb0-3409ea954787 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:34.336814 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75066e8b-e801-4c52-bfb0-3409ea954787 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:34.337095 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75066e8b-e801-4c52-bfb0-3409ea954787 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:34.338062 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75066e8b-e801-4c52-bfb0-3409ea954787 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:34.338370 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75066e8b-e801-4c52-bfb0-3409ea954787 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:34.340509 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75066e8b-e801-4c52-bfb0-3409ea954787 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:34.357406 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 439/873] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:34.361053 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75066e8b-e801-4c52-bfb0-3409ea954787 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:34.377513 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 435/874] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:34 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:15:34.378628 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da172402-0d51-45d7-9782-8018f17a8dcc None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:34.470280 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75066e8b-e801-4c52-bfb0-3409ea954787 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:34.473737 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:34.509316 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882079a-17f8-441d-83d5-4698ef46daf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:34.511013 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882079a-17f8-441d-83d5-4698ef46daf8 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:34.511634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882079a-17f8-441d-83d5-4698ef46daf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:34.512227 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882079a-17f8-441d-83d5-4698ef46daf8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:34.516552 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da172402-0d51-45d7-9782-8018f17a8dcc None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:34.521060 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b882079a-17f8-441d-83d5-4698ef46daf8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:34.522754 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b882079a-17f8-441d-83d5-4698ef46daf8 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:34.537072 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-b882079a-17f8-441d-83d5-4698ef46daf8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:34.537072 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:34.537072 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:34.537072 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:34.537072 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:34.537072 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:34.537072 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:34.537072 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:34.537072 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:34.537072 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:34.537072 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:34.537072 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:34.537072 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:15:34.537072 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:15:34.537072 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:15:34.538019 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:15:34.538019 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:34.538019 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:34.538019 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:34.538019 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:34.538019 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:34.538019 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:34.538019 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:34.538019 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:34.538019 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:34.538019 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:34.538019 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:34.538019 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:34.538019 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:34.538019 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:34.538019 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:34.538019 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:34.539028 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:15:34.539028 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:15:34.539028 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:15:34.539028 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:15:34.539028 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:15:34.539028 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:15:34.539028 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:34.539028 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:34.542264 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 440/875] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:15:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:34.547913 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882079a-17f8-441d-83d5-4698ef46daf8 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:34.561117 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83de76b-ece2-4cd9-a956-b1ebe2d72d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:34.562205 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83de76b-ece2-4cd9-a956-b1ebe2d72d14 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:34.562556 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83de76b-ece2-4cd9-a956-b1ebe2d72d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:34.562900 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83de76b-ece2-4cd9-a956-b1ebe2d72d14 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:34.563554 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83de76b-ece2-4cd9-a956-b1ebe2d72d14 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:34.565127 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83de76b-ece2-4cd9-a956-b1ebe2d72d14 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:34.575365 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-f83de76b-ece2-4cd9-a956-b1ebe2d72d14 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 20:15:34.575365 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:34.575365 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:15:34.575365 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:15:34.575365 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:15:34.575365 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:15:34.575365 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:15:34.575365 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:34.575365 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 20:15:34.575365 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:34.575365 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:15:34.575365 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:34.575365 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:34.575365 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:34.575365 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:34.575365 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:34.575365 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:34.576433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:34.576433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:34.576433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:34.576433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:34.576433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:34.576433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:34.576433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:15:34.576433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:15:34.576433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:15:34.576433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:15:34.576433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:34.576433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:34.576433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:34.576433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:34.576433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:34.576433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:34.577337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:34.577337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:34.577337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:34.577337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:34.577337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:34.577337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:34.577337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:34.577337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:34.577337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:34.577337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:34.577337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:15:34.577337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:34.577337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 20:15:34.577337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:34.577337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 436/876] 10.222.0.26 () {56 vars in 1126 bytes} [Fri Sep 9 20:15:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:34.582740 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83de76b-ece2-4cd9-a956-b1ebe2d72d14 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:34.617942 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:34.688875 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882079a-17f8-441d-83d5-4698ef46daf8 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:34.704281 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83de76b-ece2-4cd9-a956-b1ebe2d72d14 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:34.730466 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85698a00-3d8e-415e-bd80-bf94bd75a23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:34.731360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85698a00-3d8e-415e-bd80-bf94bd75a23b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:34.731873 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85698a00-3d8e-415e-bd80-bf94bd75a23b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:34.732607 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85698a00-3d8e-415e-bd80-bf94bd75a23b None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:34.733919 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85698a00-3d8e-415e-bd80-bf94bd75a23b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:34.734625 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85698a00-3d8e-415e-bd80-bf94bd75a23b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:34.736251 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85698a00-3d8e-415e-bd80-bf94bd75a23b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:34.738098 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff585f6-986a-4a8c-afb8-8a26ae9a7068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:34.738700 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff585f6-986a-4a8c-afb8-8a26ae9a7068 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:34.738975 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff585f6-986a-4a8c-afb8-8a26ae9a7068 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:34.739246 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff585f6-986a-4a8c-afb8-8a26ae9a7068 None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:34.739913 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff585f6-986a-4a8c-afb8-8a26ae9a7068 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:34.740208 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff585f6-986a-4a8c-afb8-8a26ae9a7068 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:34.741580 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff585f6-986a-4a8c-afb8-8a26ae9a7068 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:34.742599 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:34.750991 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 441/877] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:15:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:34.751569 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 437/878] 10.222.0.26 () {56 vars in 1122 bytes} [Fri Sep 9 20:15:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:34.752399 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85698a00-3d8e-415e-bd80-bf94bd75a23b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:34.752789 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff585f6-986a-4a8c-afb8-8a26ae9a7068 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:34.859784 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff585f6-986a-4a8c-afb8-8a26ae9a7068 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:34.861089 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85698a00-3d8e-415e-bd80-bf94bd75a23b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:34.873478 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:34.897316 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7dca9a-d1d4-4742-9353-730c87807a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:34.897925 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7dca9a-d1d4-4742-9353-730c87807a4a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:34.898207 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7dca9a-d1d4-4742-9353-730c87807a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:34.898480 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7dca9a-d1d4-4742-9353-730c87807a4a None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:34.899103 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7dca9a-d1d4-4742-9353-730c87807a4a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:34.899406 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7dca9a-d1d4-4742-9353-730c87807a4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:34.900763 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7dca9a-d1d4-4742-9353-730c87807a4a None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:34.901163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb197b77-079a-474e-8ae1-927b5a3b1b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:34.901817 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb197b77-079a-474e-8ae1-927b5a3b1b46 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:34.902096 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb197b77-079a-474e-8ae1-927b5a3b1b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:34.902371 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb197b77-079a-474e-8ae1-927b5a3b1b46 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:34.903017 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb197b77-079a-474e-8ae1-927b5a3b1b46 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:34.904351 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb197b77-079a-474e-8ae1-927b5a3b1b46 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:34.905470 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 438/879] 10.222.0.26 () {56 vars in 1122 bytes} [Fri Sep 9 20:15:34 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:34.906088 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7dca9a-d1d4-4742-9353-730c87807a4a None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:34.929586 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 442/880] 10.222.0.26 () {60 vars in 1141 bytes} [Fri Sep 9 20:15:34 2022] POST /identity/v3/services => generated 269 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:15:34.995247 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:35.050437 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7dca9a-d1d4-4742-9353-730c87807a4a None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:35.099937 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d764d33-6d04-4afe-8a1f-54b9eb74f3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:35.100556 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d764d33-6d04-4afe-8a1f-54b9eb74f3a2 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:35.100833 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d764d33-6d04-4afe-8a1f-54b9eb74f3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:35.101104 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d764d33-6d04-4afe-8a1f-54b9eb74f3a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:35.101952 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d764d33-6d04-4afe-8a1f-54b9eb74f3a2 None admin] RBAC: Adding query filter params (role.id=8ec32f702e0c447993ff294cbac536b1, scope.project.id=933820e2adbc473bbd57a4d59be75d72, user.id=cedaf156c5584d62beedde598c94f322) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:35.102948 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d764d33-6d04-4afe-8a1f-54b9eb74f3a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:35.104446 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d764d33-6d04-4afe-8a1f-54b9eb74f3a2 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:35.111533 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 439/881] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:15:34 2022] GET /identity/v3/role_assignments?user.id=cedaf156c5584d62beedde598c94f322&scope.project.id=933820e2adbc473bbd57a4d59be75d72&role.id=8ec32f702e0c447993ff294cbac536b1 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:35.765820 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678ed16-6379-4d4d-a825-cba71d3963a7 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:35.766364 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678ed16-6379-4d4d-a825-cba71d3963a7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:35.766364 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678ed16-6379-4d4d-a825-cba71d3963a7 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:35.767912 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 443/882] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:35.773873 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997fded-527a-4b99-826c-9b41d7c14cb3 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:35.774287 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997fded-527a-4b99-826c-9b41d7c14cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:35.774475 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997fded-527a-4b99-826c-9b41d7c14cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:35.792326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-f997fded-527a-4b99-826c-9b41d7c14cb3 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:35.793510 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-f997fded-527a-4b99-826c-9b41d7c14cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:35.860837 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 440/883] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:35 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:15:36.134691 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678ed16-6379-4d4d-a825-cba71d3963a7 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:36.246859 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678ed16-6379-4d4d-a825-cba71d3963a7 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:36.284883 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3de24f-ea47-485f-b832-0665b3f8fb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:36.285935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3de24f-ea47-485f-b832-0665b3f8fb42 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:36.286458 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3de24f-ea47-485f-b832-0665b3f8fb42 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:36.286946 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3de24f-ea47-485f-b832-0665b3f8fb42 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:36.287741 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3de24f-ea47-485f-b832-0665b3f8fb42 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:36.289392 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3de24f-ea47-485f-b832-0665b3f8fb42 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:36.298751 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-cb3de24f-ea47-485f-b832-0665b3f8fb42 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 20:15:36.298751 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:36.298751 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:15:36.298751 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:15:36.298751 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:15:36.298751 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:15:36.298751 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:15:36.298751 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:36.298751 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 20:15:36.298751 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:36.298751 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:15:36.298751 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:36.298751 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:36.298751 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:36.298751 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:36.298751 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:36.298751 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:36.299789 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:36.299789 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:36.299789 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:36.299789 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:36.299789 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:36.299789 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:36.299789 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:15:36.299789 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:15:36.299789 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:15:36.299789 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:15:36.299789 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:36.299789 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:36.299789 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:36.299789 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:36.299789 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:36.299789 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:36.300664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:36.300664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:36.300664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:36.300664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:36.300664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:36.300664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:36.300664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:36.300664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:36.300664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:36.300664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:36.300664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:15:36.300664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:36.300664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 20:15:36.300664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:36.303928 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 444/884] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Sep 9 20:15:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:36.311258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f997fded-527a-4b99-826c-9b41d7c14cb3 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:36.360287 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:36.408731 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f997fded-527a-4b99-826c-9b41d7c14cb3 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:36.451270 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a49a64-7137-48cd-9fb1-a00026d31b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:36.451875 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a49a64-7137-48cd-9fb1-a00026d31b77 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:36.452152 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a49a64-7137-48cd-9fb1-a00026d31b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:36.452424 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a49a64-7137-48cd-9fb1-a00026d31b77 None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:36.453048 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a49a64-7137-48cd-9fb1-a00026d31b77 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:36.453372 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a49a64-7137-48cd-9fb1-a00026d31b77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:36.454728 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a49a64-7137-48cd-9fb1-a00026d31b77 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:36.459788 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 441/885] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Sep 9 20:15:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:36.465104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3de24f-ea47-485f-b832-0665b3f8fb42 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:36.500498 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:36.570487 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3de24f-ea47-485f-b832-0665b3f8fb42 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:36.610023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db7f97-8eb9-4461-aeca-3b69a41d454b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:36.611002 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db7f97-8eb9-4461-aeca-3b69a41d454b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:36.611490 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db7f97-8eb9-4461-aeca-3b69a41d454b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:36.611990 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db7f97-8eb9-4461-aeca-3b69a41d454b None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:36.612834 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db7f97-8eb9-4461-aeca-3b69a41d454b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:36.613410 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db7f97-8eb9-4461-aeca-3b69a41d454b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:36.615588 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db7f97-8eb9-4461-aeca-3b69a41d454b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:36.621112 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 445/886] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Sep 9 20:15:36 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:36.630534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a49a64-7137-48cd-9fb1-a00026d31b77 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:36.728705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a49a64-7137-48cd-9fb1-a00026d31b77 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:36.745886 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:36.764272 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6ee538-8ee0-44a0-b37d-da01ca604104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:36.764878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ee538-8ee0-44a0-b37d-da01ca604104 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:36.765165 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ee538-8ee0-44a0-b37d-da01ca604104 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:36.765438 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ee538-8ee0-44a0-b37d-da01ca604104 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:36.766128 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6ee538-8ee0-44a0-b37d-da01ca604104 None admin] RBAC: Adding query filter params (interface=public, service_id=3f0ddd2f200147f4ab238b87bb0f857b, region_id=RegionOne) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:36.766434 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6ee538-8ee0-44a0-b37d-da01ca604104 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:36.767783 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6ee538-8ee0-44a0-b37d-da01ca604104 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:36.773268 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 442/887] 10.222.0.26 () {56 vars in 1251 bytes} [Fri Sep 9 20:15:36 2022] GET /identity/v3/endpoints?service_id=3f0ddd2f200147f4ab238b87bb0f857b&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:36.778978 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db7f97-8eb9-4461-aeca-3b69a41d454b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:36.880773 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:36.885494 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db7f97-8eb9-4461-aeca-3b69a41d454b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:36.923001 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebda549-5a81-467c-97c9-76ac89996cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:36.923530 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebda549-5a81-467c-97c9-76ac89996cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:36.923814 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebda549-5a81-467c-97c9-76ac89996cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:36.924090 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebda549-5a81-467c-97c9-76ac89996cf6 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:36.924842 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebda549-5a81-467c-97c9-76ac89996cf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:36.926242 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebda549-5a81-467c-97c9-76ac89996cf6 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:36.932580 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 446/888] 10.222.0.26 () {56 vars in 1087 bytes} [Fri Sep 9 20:15:36 2022] GET /identity/v3/services => generated 1268 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:15:37.175054 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa6114-b1d4-494e-9985-cefe62b87685 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:37.175559 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa6114-b1d4-494e-9985-cefe62b87685 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:37.175559 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa6114-b1d4-494e-9985-cefe62b87685 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:37.177610 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 443/889] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:37.182880 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134e94e-a4d6-4ab4-a96d-7b4a6562d460 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:37.183171 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134e94e-a4d6-4ab4-a96d-7b4a6562d460 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:37.183450 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134e94e-a4d6-4ab4-a96d-7b4a6562d460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:37.202021 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-3134e94e-a4d6-4ab4-a96d-7b4a6562d460 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:37.203201 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3134e94e-a4d6-4ab4-a96d-7b4a6562d460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:37.271365 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 447/890] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:37 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:15:37.281805 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97523d3f-d9f2-42e8-8490-a5a985f518ec None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:37.282185 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97523d3f-d9f2-42e8-8490-a5a985f518ec None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:37.282533 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97523d3f-d9f2-42e8-8490-a5a985f518ec None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:37.284354 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 444/891] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:37.290998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31c766-a01d-4c69-b1f7-b73acf1f4f24 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:37.291342 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31c766-a01d-4c69-b1f7-b73acf1f4f24 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:37.291568 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31c766-a01d-4c69-b1f7-b73acf1f4f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:37.312040 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-bd31c766-a01d-4c69-b1f7-b73acf1f4f24 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:37.313331 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-bd31c766-a01d-4c69-b1f7-b73acf1f4f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:37.389363 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 448/892] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:37 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:15:37.411013 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e91ee-375d-43bd-bd0f-7fa046f5e8d1 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:37.411884 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e91ee-375d-43bd-bd0f-7fa046f5e8d1 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:37.412167 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e91ee-375d-43bd-bd0f-7fa046f5e8d1 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:37.413815 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 445/893] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:37.421271 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2090ce-1d4c-43e9-a8c4-11d91c2a2d4b None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:37.421271 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2090ce-1d4c-43e9-a8c4-11d91c2a2d4b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:37.421271 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2090ce-1d4c-43e9-a8c4-11d91c2a2d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:37.440690 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-7a2090ce-1d4c-43e9-a8c4-11d91c2a2d4b None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:37.442108 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7a2090ce-1d4c-43e9-a8c4-11d91c2a2d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:37.460311 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:37.518443 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 449/894] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:37 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:15:37.546551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e91ee-375d-43bd-bd0f-7fa046f5e8d1 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:37.655314 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e91ee-375d-43bd-bd0f-7fa046f5e8d1 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:37.688698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2090ce-1d4c-43e9-a8c4-11d91c2a2d4b None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:37.691122 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2674f7-4021-4e9c-986d-6294b5422190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:37.691701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2674f7-4021-4e9c-986d-6294b5422190 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:37.691977 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2674f7-4021-4e9c-986d-6294b5422190 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:37.692249 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2674f7-4021-4e9c-986d-6294b5422190 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:37.693450 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2674f7-4021-4e9c-986d-6294b5422190 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:37.694805 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2674f7-4021-4e9c-986d-6294b5422190 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:37.703751 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-1b2674f7-4021-4e9c-986d-6294b5422190 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 20:15:37.703751 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:37.703751 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:15:37.703751 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:15:37.703751 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:15:37.703751 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:15:37.703751 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:15:37.703751 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:37.703751 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 20:15:37.703751 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:37.703751 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:15:37.703751 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:37.703751 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:37.703751 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:37.703751 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:37.703751 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:37.704734 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:37.704734 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:37.704734 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:37.704734 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:37.704734 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:37.704734 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:37.704734 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:37.704734 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:15:37.704734 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:15:37.704734 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:15:37.704734 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:15:37.704734 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:37.704734 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:37.704734 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:37.704734 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:37.704734 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:37.704734 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:37.706237 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:37.706237 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:37.706237 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:37.706237 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:37.706237 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:37.706237 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:37.706237 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:37.706237 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:37.706237 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:37.706237 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:37.706237 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:15:37.706237 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:37.706237 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 20:15:37.706237 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:37.706237 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 446/895] 10.222.0.26 () {56 vars in 1132 bytes} [Fri Sep 9 20:15:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:37.710680 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2674f7-4021-4e9c-986d-6294b5422190 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:37.757321 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:37.813099 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2674f7-4021-4e9c-986d-6294b5422190 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:37.820179 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2090ce-1d4c-43e9-a8c4-11d91c2a2d4b None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:37.847653 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec575d-d96d-4066-99f5-d858bca85a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:37.848246 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec575d-d96d-4066-99f5-d858bca85a01 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:37.848519 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec575d-d96d-4066-99f5-d858bca85a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:37.848838 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec575d-d96d-4066-99f5-d858bca85a01 None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:37.849492 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec575d-d96d-4066-99f5-d858bca85a01 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:37.849794 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec575d-d96d-4066-99f5-d858bca85a01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:37.851147 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec575d-d96d-4066-99f5-d858bca85a01 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:37.856843 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91385926-788c-44de-9e4f-a91c5f87dc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:37.857485 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91385926-788c-44de-9e4f-a91c5f87dc8c None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:37.857812 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91385926-788c-44de-9e4f-a91c5f87dc8c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:37.858084 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91385926-788c-44de-9e4f-a91c5f87dc8c None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:37.858738 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91385926-788c-44de-9e4f-a91c5f87dc8c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:37.860386 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 447/896] 10.222.0.26 () {56 vars in 1126 bytes} [Fri Sep 9 20:15:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:37.860494 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91385926-788c-44de-9e4f-a91c5f87dc8c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:37.861020 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec575d-d96d-4066-99f5-d858bca85a01 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:37.876935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 450/897] 10.222.0.26 () {60 vars in 1141 bytes} [Fri Sep 9 20:15:37 2022] POST /identity/v3/services => generated 241 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:15:37.882339 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91385926-788c-44de-9e4f-a91c5f87dc8c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:37.905016 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:38.004141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec575d-d96d-4066-99f5-d858bca85a01 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:38.016040 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:38.028811 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91385926-788c-44de-9e4f-a91c5f87dc8c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:38.049475 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4639729-a2a8-42bc-ba8d-b695ae1d4efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:38.050277 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4639729-a2a8-42bc-ba8d-b695ae1d4efc None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:38.050664 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4639729-a2a8-42bc-ba8d-b695ae1d4efc None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:38.051146 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4639729-a2a8-42bc-ba8d-b695ae1d4efc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:38.063928 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4639729-a2a8-42bc-ba8d-b695ae1d4efc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:15:38.069111 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-c4639729-a2a8-42bc-ba8d-b695ae1d4efc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 20:15:38.069111 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:38.069111 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:38.069111 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:38.069111 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:38.069111 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:38.069111 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:38.069111 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:38.069111 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:38.069111 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:38.069111 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:38.069111 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:38.069111 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:15:38.069111 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:15:38.069111 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:15:38.069111 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:15:38.070302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:15:38.070302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:15:38.070302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:15:38.070302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:15:38.070302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:38.070302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:38.070302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:38.070302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:38.070302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:38.070302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:38.070302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:38.070302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:38.070302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:38.070302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:38.070302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:38.070302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:38.071314 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:38.071314 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:38.071314 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:38.071314 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:38.071314 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:15:38.071314 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:15:38.071314 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:15:38.071314 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:15:38.071314 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:15:38.071314 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:15:38.071314 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 20:15:38.071314 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:38.071314 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 448/898] 10.222.0.26 () {56 vars in 1120 bytes} [Fri Sep 9 20:15:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:38.076256 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd3cf7b-c57c-4c64-9eb4-d6aec1b2ece8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:38.076876 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3cf7b-c57c-4c64-9eb4-d6aec1b2ece8 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:38.077157 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3cf7b-c57c-4c64-9eb4-d6aec1b2ece8 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:38.077521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3cf7b-c57c-4c64-9eb4-d6aec1b2ece8 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:38.078160 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd3cf7b-c57c-4c64-9eb4-d6aec1b2ece8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:38.078464 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd3cf7b-c57c-4c64-9eb4-d6aec1b2ece8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:38.078681 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4639729-a2a8-42bc-ba8d-b695ae1d4efc None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:38.079820 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd3cf7b-c57c-4c64-9eb4-d6aec1b2ece8 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:38.085974 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 451/899] 10.222.0.26 () {56 vars in 1126 bytes} [Fri Sep 9 20:15:37 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:38.091409 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd3cf7b-c57c-4c64-9eb4-d6aec1b2ece8 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:38.174664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:38.207295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4639729-a2a8-42bc-ba8d-b695ae1d4efc None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:38.210188 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd3cf7b-c57c-4c64-9eb4-d6aec1b2ece8 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:38.250869 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974d7d0-ad59-4331-a5c3-da08cc57b8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:38.251661 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974d7d0-ad59-4331-a5c3-da08cc57b8db None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:38.252041 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974d7d0-ad59-4331-a5c3-da08cc57b8db None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:38.252414 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974d7d0-ad59-4331-a5c3-da08cc57b8db None admin] PATH_INFO: `/v3/endpoints` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:38.252678 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31aa056-39b7-4949-9840-02954d7d887c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:38.253412 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9974d7d0-ad59-4331-a5c3-da08cc57b8db None admin] RBAC: Adding query filter params (interface=public, service_id=1a4494a928be4ca49bdfb81b03cf447a, region_id=RegionOne) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:38.253878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31aa056-39b7-4949-9840-02954d7d887c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:38.254233 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9974d7d0-ad59-4331-a5c3-da08cc57b8db None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:38.254501 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31aa056-39b7-4949-9840-02954d7d887c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:38.254937 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31aa056-39b7-4949-9840-02954d7d887c None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:38.255737 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31aa056-39b7-4949-9840-02954d7d887c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:38.255906 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9974d7d0-ad59-4331-a5c3-da08cc57b8db None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:38.256322 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31aa056-39b7-4949-9840-02954d7d887c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:38.258170 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31aa056-39b7-4949-9840-02954d7d887c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:38.269811 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 452/900] 10.222.0.26 () {56 vars in 1251 bytes} [Fri Sep 9 20:15:38 2022] GET /identity/v3/endpoints?service_id=1a4494a928be4ca49bdfb81b03cf447a&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:38.271282 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974d7d0-ad59-4331-a5c3-da08cc57b8db None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:38.272092 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 449/901] 10.222.0.26 () {56 vars in 1115 bytes} [Fri Sep 9 20:15:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:38.277959 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31aa056-39b7-4949-9840-02954d7d887c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:38.311725 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:38.384712 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31aa056-39b7-4949-9840-02954d7d887c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:38.401714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974d7d0-ad59-4331-a5c3-da08cc57b8db None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:38.420408 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd680f45-a327-479d-9cd5-a90e6e301734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:38.421048 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd680f45-a327-479d-9cd5-a90e6e301734 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:38.421420 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd680f45-a327-479d-9cd5-a90e6e301734 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:38.421795 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd680f45-a327-479d-9cd5-a90e6e301734 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:38.433006 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd680f45-a327-479d-9cd5-a90e6e301734 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:38.434709 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd680f45-a327-479d-9cd5-a90e6e301734 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:38.444484 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2a6c3-7290-4cbe-813c-9cc3b38695a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:38.445231 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2a6c3-7290-4cbe-813c-9cc3b38695a4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:38.445631 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2a6c3-7290-4cbe-813c-9cc3b38695a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:38.446004 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2a6c3-7290-4cbe-813c-9cc3b38695a4 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:38.446685 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd2a6c3-7290-4cbe-813c-9cc3b38695a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:38.448089 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd2a6c3-7290-4cbe-813c-9cc3b38695a4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:38.450087 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:38.454556 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-fd680f45-a327-479d-9cd5-a90e6e301734 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:38.454556 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:38.454556 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:38.454556 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:38.454556 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:38.454556 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:38.454556 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:38.454556 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:38.454556 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:38.454556 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:38.454556 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:38.454556 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:38.454556 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:38.454556 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:38.454556 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:38.454556 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:38.456878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:38.456878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:38.456878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:38.456878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:38.456878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:38.456878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:38.456878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:38.456878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:38.456878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:38.456878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:38.456878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:38.456878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:38.456878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:38.456878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:38.456878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:38.456878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:38.456878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:38.457823 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:38.457823 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:38.457823 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:38.458542 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 453/902] 10.222.0.26 () {56 vars in 1087 bytes} [Fri Sep 9 20:15:38 2022] GET /identity/v3/services => generated 1497 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:15:38.459206 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 450/903] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:15:38.469323 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2a6c3-7290-4cbe-813c-9cc3b38695a4 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:38.577912 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2a6c3-7290-4cbe-813c-9cc3b38695a4 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:38.616988 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f19e51a-a08c-4aeb-8090-b999cee2ce3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:38.617671 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19e51a-a08c-4aeb-8090-b999cee2ce3d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:38.617948 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19e51a-a08c-4aeb-8090-b999cee2ce3d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:38.618217 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19e51a-a08c-4aeb-8090-b999cee2ce3d None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:38.618875 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f19e51a-a08c-4aeb-8090-b999cee2ce3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:38.619174 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f19e51a-a08c-4aeb-8090-b999cee2ce3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:38.620551 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f19e51a-a08c-4aeb-8090-b999cee2ce3d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:38.636768 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 454/904] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:38.642656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd680f45-a327-479d-9cd5-a90e6e301734 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:38.737347 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:38.757778 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd680f45-a327-479d-9cd5-a90e6e301734 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:38.793490 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a8dd46-d7c3-4927-9a49-40aaa6b1cd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:38.794474 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8dd46-d7c3-4927-9a49-40aaa6b1cd78 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:38.794949 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8dd46-d7c3-4927-9a49-40aaa6b1cd78 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:38.795411 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8dd46-d7c3-4927-9a49-40aaa6b1cd78 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:38.803512 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a8dd46-d7c3-4927-9a49-40aaa6b1cd78 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:38.805183 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a8dd46-d7c3-4927-9a49-40aaa6b1cd78 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:38.816629 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-89a8dd46-d7c3-4927-9a49-40aaa6b1cd78 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 20:15:38.816629 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:38.816629 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:38.816629 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:38.816629 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:38.816629 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:38.816629 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:38.816629 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:38.816629 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:38.816629 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:38.816629 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:38.816629 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:38.816629 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:15:38.816629 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:15:38.816629 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:15:38.816629 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:15:38.818156 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:38.818156 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:38.818156 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:15:38.818156 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:38.818156 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:15:38.818156 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:38.818156 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:38.818156 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:38.818156 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:38.818156 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:38.818156 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:38.818156 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:38.818156 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:38.818156 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:38.818156 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:38.818156 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:38.818156 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:38.818156 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:38.819131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:38.819131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:38.819131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:15:38.819131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:15:38.819131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:15:38.819131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:15:38.819131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:15:38.819131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:15:38.819131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 20:15:38.819131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:38.821709 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 451/905] 10.222.0.26 () {56 vars in 1117 bytes} [Fri Sep 9 20:15:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:38.832680 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f19e51a-a08c-4aeb-8090-b999cee2ce3d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:38.898458 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:38.947008 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f19e51a-a08c-4aeb-8090-b999cee2ce3d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:38.982548 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56adf35b-4fb1-46ab-9241-c5e94f46477a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:38.983154 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56adf35b-4fb1-46ab-9241-c5e94f46477a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:38.983429 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56adf35b-4fb1-46ab-9241-c5e94f46477a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:38.983696 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56adf35b-4fb1-46ab-9241-c5e94f46477a None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:38.984476 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56adf35b-4fb1-46ab-9241-c5e94f46477a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:38.984776 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56adf35b-4fb1-46ab-9241-c5e94f46477a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:38.986252 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56adf35b-4fb1-46ab-9241-c5e94f46477a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:39.006296 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 455/906] 10.222.0.26 () {56 vars in 1149 bytes} [Fri Sep 9 20:15:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:39.011833 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a8dd46-d7c3-4927-9a49-40aaa6b1cd78 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:39.126200 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a8dd46-d7c3-4927-9a49-40aaa6b1cd78 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:39.163647 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c17c7b-2d46-4089-93d2-3549dc333c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:39.164587 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c17c7b-2d46-4089-93d2-3549dc333c06 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:39.165084 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c17c7b-2d46-4089-93d2-3549dc333c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:39.165596 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c17c7b-2d46-4089-93d2-3549dc333c06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:39.185948 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:39.214836 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3cd2c3-f95c-448d-a720-bf5bb31843f9 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:39.215148 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3cd2c3-f95c-448d-a720-bf5bb31843f9 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:39.215398 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3cd2c3-f95c-448d-a720-bf5bb31843f9 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:39.218170 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 456/907] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:39.222466 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472e7b1-dc9f-4434-8a44-aa12b35007a0 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:39.222859 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472e7b1-dc9f-4434-8a44-aa12b35007a0 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:39.223144 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472e7b1-dc9f-4434-8a44-aa12b35007a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:39.242697 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c17c7b-2d46-4089-93d2-3549dc333c06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:39.243531 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-c472e7b1-dc9f-4434-8a44-aa12b35007a0 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:39.244168 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c17c7b-2d46-4089-93d2-3549dc333c06 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:39.244935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c472e7b1-dc9f-4434-8a44-aa12b35007a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:39.333343 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 457/908] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:15:39.358342 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 452/909] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:15:39 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:15:39.363667 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472e7b1-dc9f-4434-8a44-aa12b35007a0 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:39.413147 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:39.470913 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472e7b1-dc9f-4434-8a44-aa12b35007a0 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:39.506674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7197685-4efd-4fe7-aa62-14899215d9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:39.507302 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7197685-4efd-4fe7-aa62-14899215d9df None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:39.507578 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7197685-4efd-4fe7-aa62-14899215d9df None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:39.507847 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7197685-4efd-4fe7-aa62-14899215d9df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:39.517942 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7197685-4efd-4fe7-aa62-14899215d9df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:39.519356 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7197685-4efd-4fe7-aa62-14899215d9df None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:39.537325 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-e7197685-4efd-4fe7-aa62-14899215d9df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:39.537325 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:39.537325 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:39.537325 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:39.537325 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:39.537325 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:39.537325 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:39.537325 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:39.537325 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:39.537325 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:39.537325 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:39.537325 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:39.537325 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:39.537325 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:39.537325 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:39.537325 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:39.539104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:39.539104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:39.539104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:39.539104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:39.539104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:39.539104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:39.539104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:39.539104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:39.539104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:39.539104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:39.539104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:39.539104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:39.539104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:39.539104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:39.539104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:39.539104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:39.539104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:39.540616 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:39.540616 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:39.540616 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:39.540616 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 458/910] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:39.546337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c17c7b-2d46-4089-93d2-3549dc333c06 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:39.591160 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:39.600397 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7197685-4efd-4fe7-aa62-14899215d9df None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:39.668197 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c17c7b-2d46-4089-93d2-3549dc333c06 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:39.717796 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266cac8-94c8-46fa-87f3-53751fa5f6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:39.718415 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266cac8-94c8-46fa-87f3-53751fa5f6b7 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:39.718696 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266cac8-94c8-46fa-87f3-53751fa5f6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:39.718970 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266cac8-94c8-46fa-87f3-53751fa5f6b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:39.719613 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4266cac8-94c8-46fa-87f3-53751fa5f6b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:39.720071 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4266cac8-94c8-46fa-87f3-53751fa5f6b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:39.721833 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4266cac8-94c8-46fa-87f3-53751fa5f6b7 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:39.730386 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:39.734563 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7197685-4efd-4fe7-aa62-14899215d9df None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:39.740372 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 453/911] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:39.745465 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266cac8-94c8-46fa-87f3-53751fa5f6b7 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:39.778521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c0e91-069d-4e6c-b91b-cf0eadc3147f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:39.779229 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c0e91-069d-4e6c-b91b-cf0eadc3147f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:39.779600 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c0e91-069d-4e6c-b91b-cf0eadc3147f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:39.779939 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c0e91-069d-4e6c-b91b-cf0eadc3147f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:39.780592 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c0e91-069d-4e6c-b91b-cf0eadc3147f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:39.782045 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c0e91-069d-4e6c-b91b-cf0eadc3147f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:39.797193 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-e36c0e91-069d-4e6c-b91b-cf0eadc3147f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 20:15:39.797193 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:39.797193 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:15:39.797193 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:15:39.797193 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:15:39.797193 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:15:39.797193 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:15:39.797193 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:39.797193 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 20:15:39.797193 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:39.797193 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:15:39.797193 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:39.797193 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:39.797193 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:39.797193 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:39.797193 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:39.797193 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:39.798334 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:39.798334 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:39.798334 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:39.798334 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:39.798334 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:39.798334 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:39.798334 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:15:39.798334 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:15:39.798334 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:15:39.798334 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:15:39.798334 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:39.798334 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:39.798334 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:39.798334 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:39.798334 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:39.798334 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:39.802730 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:39.802730 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:39.802730 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:39.802730 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:39.802730 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:39.802730 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:39.802730 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:39.802730 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:39.802730 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:39.802730 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:39.802730 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:15:39.802730 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:39.802730 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 20:15:39.802730 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:39.802730 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 459/912] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Sep 9 20:15:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:15:39.804859 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c0e91-069d-4e6c-b91b-cf0eadc3147f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:39.862916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266cac8-94c8-46fa-87f3-53751fa5f6b7 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:39.865378 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:39.909403 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14a200-ce7f-4ff2-8898-1b008151f32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:39.910632 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14a200-ce7f-4ff2-8898-1b008151f32e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:39.911973 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14a200-ce7f-4ff2-8898-1b008151f32e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:39.912475 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14a200-ce7f-4ff2-8898-1b008151f32e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:39.922439 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14a200-ce7f-4ff2-8898-1b008151f32e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:39.924799 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14a200-ce7f-4ff2-8898-1b008151f32e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:39.927010 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c0e91-069d-4e6c-b91b-cf0eadc3147f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:39.940775 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-5a14a200-ce7f-4ff2-8898-1b008151f32e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:39.940775 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:39.940775 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:39.940775 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:39.940775 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:39.940775 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:39.940775 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:39.940775 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:39.940775 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:39.940775 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:39.940775 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:39.940775 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:39.940775 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:15:39.940775 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:15:39.940775 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:15:39.941800 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:15:39.941800 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:39.941800 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:39.941800 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:39.941800 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:39.941800 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:39.941800 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:39.941800 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:39.941800 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:39.941800 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:39.941800 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:39.941800 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:39.941800 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:39.941800 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:39.941800 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:39.941800 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:39.941800 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:39.942772 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:15:39.942772 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:15:39.942772 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:15:39.942772 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:15:39.942772 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:15:39.942772 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:15:39.942772 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:39.942772 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:39.945381 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 454/913] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:15:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:15:39.954203 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14a200-ce7f-4ff2-8898-1b008151f32e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:39.970804 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce02a71-b3ab-42bf-8e27-20a8d1a93397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:39.971418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce02a71-b3ab-42bf-8e27-20a8d1a93397 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:39.971702 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce02a71-b3ab-42bf-8e27-20a8d1a93397 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:39.972117 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce02a71-b3ab-42bf-8e27-20a8d1a93397 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:39.972660 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce02a71-b3ab-42bf-8e27-20a8d1a93397 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:39.972955 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce02a71-b3ab-42bf-8e27-20a8d1a93397 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:39.974892 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce02a71-b3ab-42bf-8e27-20a8d1a93397 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:39.979959 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 460/914] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Sep 9 20:15:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:39.986299 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce02a71-b3ab-42bf-8e27-20a8d1a93397 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:40.017081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:40.091796 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14a200-ce7f-4ff2-8898-1b008151f32e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:40.118503 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce02a71-b3ab-42bf-8e27-20a8d1a93397 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:40.136731 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f63544-b391-4636-b5cc-a559cdf6974f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:40.137633 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f63544-b391-4636-b5cc-a559cdf6974f None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:40.137912 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f63544-b391-4636-b5cc-a559cdf6974f None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:40.138181 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f63544-b391-4636-b5cc-a559cdf6974f None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:40.138863 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f63544-b391-4636-b5cc-a559cdf6974f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:40.139160 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f63544-b391-4636-b5cc-a559cdf6974f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:40.141075 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f63544-b391-4636-b5cc-a559cdf6974f None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:40.161733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 455/915] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:15:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:40.163850 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f63544-b391-4636-b5cc-a559cdf6974f None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:40.164151 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ecd95-4993-463f-9bd8-631f73229358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:40.164925 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ecd95-4993-463f-9bd8-631f73229358 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:40.165208 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ecd95-4993-463f-9bd8-631f73229358 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:40.165513 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ecd95-4993-463f-9bd8-631f73229358 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:40.166190 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9ecd95-4993-463f-9bd8-631f73229358 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:40.166492 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9ecd95-4993-463f-9bd8-631f73229358 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:40.167839 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9ecd95-4993-463f-9bd8-631f73229358 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:40.177992 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 461/916] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Sep 9 20:15:39 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:40.181373 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ecd95-4993-463f-9bd8-631f73229358 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:40.195877 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:40.284387 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f63544-b391-4636-b5cc-a559cdf6974f None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:40.291886 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:40.301890 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ecd95-4993-463f-9bd8-631f73229358 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:40.321132 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37998819-a5ad-49b0-98fc-c784e184041a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:40.321831 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37998819-a5ad-49b0-98fc-c784e184041a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:40.322117 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37998819-a5ad-49b0-98fc-c784e184041a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:40.322392 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37998819-a5ad-49b0-98fc-c784e184041a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:40.323384 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37998819-a5ad-49b0-98fc-c784e184041a None admin] RBAC: Adding query filter params (role.id=6f8d308c22cc406ba07771a051b4bde2, scope.project.id=933820e2adbc473bbd57a4d59be75d72, user.id=cedaf156c5584d62beedde598c94f322) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:40.323567 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37998819-a5ad-49b0-98fc-c784e184041a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:40.324961 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37998819-a5ad-49b0-98fc-c784e184041a None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:40.333777 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 456/917] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:15:40 2022] GET /identity/v3/role_assignments?user.id=cedaf156c5584d62beedde598c94f322&scope.project.id=933820e2adbc473bbd57a4d59be75d72&role.id=6f8d308c22cc406ba07771a051b4bde2 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:40.335678 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b800d52-b536-4472-ae5c-b1249ba53646 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:40.336192 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b800d52-b536-4472-ae5c-b1249ba53646 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:40.336850 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b800d52-b536-4472-ae5c-b1249ba53646 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:40.337166 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b28815-129f-4281-99f1-0049ca48789c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:40.337831 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b28815-129f-4281-99f1-0049ca48789c None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:40.338101 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b28815-129f-4281-99f1-0049ca48789c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:40.338370 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b28815-129f-4281-99f1-0049ca48789c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:40.339052 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b28815-129f-4281-99f1-0049ca48789c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:40.343905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b28815-129f-4281-99f1-0049ca48789c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:40.344683 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 457/918] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:40 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:40.346588 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd58c4-5b9b-481b-85bf-173fd831c9ee None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:40.347347 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd58c4-5b9b-481b-85bf-173fd831c9ee None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:40.348262 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd58c4-5b9b-481b-85bf-173fd831c9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:40.365480 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 462/919] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:15:40 2022] POST /identity/v3/endpoints => generated 346 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:15:40.371710 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-61bd58c4-5b9b-481b-85bf-173fd831c9ee None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:40.373363 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-61bd58c4-5b9b-481b-85bf-173fd831c9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:40.465541 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 458/920] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:15:40.745868 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b28815-129f-4281-99f1-0049ca48789c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:40.854214 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b28815-129f-4281-99f1-0049ca48789c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:40.867052 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07a3cc-9e75-4bff-b096-a4f1ad15a476 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:40.867378 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07a3cc-9e75-4bff-b096-a4f1ad15a476 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:40.867717 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07a3cc-9e75-4bff-b096-a4f1ad15a476 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:40.869359 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 459/921] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:40.874666 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c175c61-e042-4c8b-a9f4-399e671c4328 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:40.875008 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c175c61-e042-4c8b-a9f4-399e671c4328 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:40.875324 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c175c61-e042-4c8b-a9f4-399e671c4328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:40.893876 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee34424-7a55-4ff6-92a2-71901ee609fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:40.894625 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee34424-7a55-4ff6-92a2-71901ee609fa None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:40.894914 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee34424-7a55-4ff6-92a2-71901ee609fa None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:40.895190 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee34424-7a55-4ff6-92a2-71901ee609fa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:40.895404 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-3c175c61-e042-4c8b-a9f4-399e671c4328 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:40.896311 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee34424-7a55-4ff6-92a2-71901ee609fa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:40.897127 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-3c175c61-e042-4c8b-a9f4-399e671c4328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:40.897709 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee34424-7a55-4ff6-92a2-71901ee609fa None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:40.905286 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:40.907691 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-1ee34424-7a55-4ff6-92a2-71901ee609fa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 20:15:40.907691 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:40.907691 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:15:40.907691 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:15:40.907691 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:15:40.907691 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:15:40.907691 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:15:40.907691 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:40.907691 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 20:15:40.907691 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:40.907691 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:15:40.907691 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:40.907691 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:40.907691 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:40.907691 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:40.907691 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:40.907691 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:40.909281 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:40.909281 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:40.909281 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:40.909281 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:40.909281 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:40.909281 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:40.909281 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:15:40.909281 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:15:40.909281 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:15:40.909281 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:15:40.909281 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:40.909281 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:40.909281 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:40.909281 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:40.909281 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:40.909281 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:40.910454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:40.910454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:40.910454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:40.910454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:40.910454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:40.910454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:40.910454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:40.910454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:40.910454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:40.910454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:40.910454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:15:40.910454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:40.910454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 20:15:40.910454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:40.913057 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 463/922] 10.222.0.26 () {56 vars in 1126 bytes} [Fri Sep 9 20:15:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:40.916809 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee34424-7a55-4ff6-92a2-71901ee609fa None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:40.933893 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:40.980309 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 460/923] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:15:41.026762 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee34424-7a55-4ff6-92a2-71901ee609fa None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:41.063630 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d95a9-04f8-4f29-b8ca-a172f53b94f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:41.064687 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d95a9-04f8-4f29-b8ca-a172f53b94f7 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:41.065331 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d95a9-04f8-4f29-b8ca-a172f53b94f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:41.065894 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d95a9-04f8-4f29-b8ca-a172f53b94f7 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:41.066856 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d95a9-04f8-4f29-b8ca-a172f53b94f7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:41.067975 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d95a9-04f8-4f29-b8ca-a172f53b94f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:41.069800 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d95a9-04f8-4f29-b8ca-a172f53b94f7 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:41.076203 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 464/924] 10.222.0.26 () {56 vars in 1122 bytes} [Fri Sep 9 20:15:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:41.083061 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c175c61-e042-4c8b-a9f4-399e671c4328 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:41.193779 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c175c61-e042-4c8b-a9f4-399e671c4328 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:41.229190 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b20bb18-79b5-4f03-9c91-5a2604614cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:41.229832 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20bb18-79b5-4f03-9c91-5a2604614cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:41.230112 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20bb18-79b5-4f03-9c91-5a2604614cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:41.230387 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20bb18-79b5-4f03-9c91-5a2604614cd4 None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:41.231017 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b20bb18-79b5-4f03-9c91-5a2604614cd4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:41.231317 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b20bb18-79b5-4f03-9c91-5a2604614cd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:41.232783 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b20bb18-79b5-4f03-9c91-5a2604614cd4 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:41.238230 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 461/925] 10.222.0.26 () {56 vars in 1122 bytes} [Fri Sep 9 20:15:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:41.244432 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d95a9-04f8-4f29-b8ca-a172f53b94f7 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:41.259905 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b20bb18-79b5-4f03-9c91-5a2604614cd4 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:41.268272 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:41.354609 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:41.380424 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d95a9-04f8-4f29-b8ca-a172f53b94f7 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:41.400714 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b20bb18-79b5-4f03-9c91-5a2604614cd4 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:41.417698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3049f05-8bfc-4eeb-ba8b-8b1d33891783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:41.418649 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3049f05-8bfc-4eeb-ba8b-8b1d33891783 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:41.419122 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3049f05-8bfc-4eeb-ba8b-8b1d33891783 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:41.419626 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3049f05-8bfc-4eeb-ba8b-8b1d33891783 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:41.420526 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3049f05-8bfc-4eeb-ba8b-8b1d33891783 None admin] RBAC: Adding query filter params (interface=public, service_id=da7590d6ed8246e6aeeaa26461e5a88d, region_id=RegionOne) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:41.421128 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3049f05-8bfc-4eeb-ba8b-8b1d33891783 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:41.422750 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3049f05-8bfc-4eeb-ba8b-8b1d33891783 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:41.433560 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 465/926] 10.222.0.26 () {56 vars in 1251 bytes} [Fri Sep 9 20:15:41 2022] GET /identity/v3/endpoints?service_id=da7590d6ed8246e6aeeaa26461e5a88d&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:41.435261 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3049f05-8bfc-4eeb-ba8b-8b1d33891783 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:41.436212 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ec65b-bfb9-41ac-8d65-52c954c293ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:41.436826 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ec65b-bfb9-41ac-8d65-52c954c293ba None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:41.437103 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ec65b-bfb9-41ac-8d65-52c954c293ba None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:41.437404 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ec65b-bfb9-41ac-8d65-52c954c293ba None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:41.437982 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ec65b-bfb9-41ac-8d65-52c954c293ba None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:41.439442 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ec65b-bfb9-41ac-8d65-52c954c293ba None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:41.449142 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-aa7ec65b-bfb9-41ac-8d65-52c954c293ba None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 20:15:41.449142 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:41.449142 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:15:41.449142 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:15:41.449142 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:15:41.449142 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:15:41.449142 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:15:41.449142 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:41.449142 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 20:15:41.449142 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:41.449142 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:15:41.449142 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:41.449142 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:41.449142 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:41.449142 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:41.449142 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:41.450253 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:41.450253 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:41.450253 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:41.450253 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:41.450253 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:41.450253 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:41.450253 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:41.450253 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:15:41.450253 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:15:41.450253 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:15:41.450253 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:15:41.450253 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:41.450253 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:41.450253 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:41.450253 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:41.450253 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:41.450253 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:41.451191 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:41.451191 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:41.451191 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:41.451191 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:41.451191 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:41.451191 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:41.451191 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:41.451191 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:41.451191 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:41.451191 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:41.451191 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:15:41.451191 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:41.451191 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 20:15:41.451191 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:41.455187 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 462/927] 10.222.0.26 () {56 vars in 1132 bytes} [Fri Sep 9 20:15:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:15:41.456228 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ec65b-bfb9-41ac-8d65-52c954c293ba None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:41.530132 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:41.557662 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3049f05-8bfc-4eeb-ba8b-8b1d33891783 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:41.564007 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ec65b-bfb9-41ac-8d65-52c954c293ba None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:41.599675 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88451d4a-65f0-45c0-90d1-32d0dfc07037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:41.600472 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83261029-1409-44b8-bcfa-dfc202a8cc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:41.600963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88451d4a-65f0-45c0-90d1-32d0dfc07037 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:41.601070 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83261029-1409-44b8-bcfa-dfc202a8cc9c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:41.601366 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83261029-1409-44b8-bcfa-dfc202a8cc9c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:41.601639 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83261029-1409-44b8-bcfa-dfc202a8cc9c None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:41.602017 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88451d4a-65f0-45c0-90d1-32d0dfc07037 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:41.602265 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83261029-1409-44b8-bcfa-dfc202a8cc9c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:41.602566 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83261029-1409-44b8-bcfa-dfc202a8cc9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:41.602898 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88451d4a-65f0-45c0-90d1-32d0dfc07037 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:41.603695 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88451d4a-65f0-45c0-90d1-32d0dfc07037 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:41.603926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83261029-1409-44b8-bcfa-dfc202a8cc9c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:41.605625 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88451d4a-65f0-45c0-90d1-32d0dfc07037 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:41.617266 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 463/928] 10.222.0.26 () {56 vars in 1126 bytes} [Fri Sep 9 20:15:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:41.617459 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 466/929] 10.222.0.26 () {56 vars in 1087 bytes} [Fri Sep 9 20:15:41 2022] GET /identity/v3/services => generated 1497 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:15:41.619097 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83261029-1409-44b8-bcfa-dfc202a8cc9c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:41.734233 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83261029-1409-44b8-bcfa-dfc202a8cc9c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:41.772667 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f630338-406a-4476-b992-7fc602a8edf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:41.773311 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f630338-406a-4476-b992-7fc602a8edf0 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:41.773592 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f630338-406a-4476-b992-7fc602a8edf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:41.773862 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f630338-406a-4476-b992-7fc602a8edf0 None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:41.774508 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f630338-406a-4476-b992-7fc602a8edf0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:41.774808 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f630338-406a-4476-b992-7fc602a8edf0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:41.776191 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f630338-406a-4476-b992-7fc602a8edf0 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:41.777694 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:41.782427 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 464/930] 10.222.0.26 () {56 vars in 1126 bytes} [Fri Sep 9 20:15:41 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:41.788724 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88451d4a-65f0-45c0-90d1-32d0dfc07037 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:41.792703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:41.901416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88451d4a-65f0-45c0-90d1-32d0dfc07037 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:41.937882 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d1a69a-0806-4932-b998-91bc4fce5ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:41.938490 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1a69a-0806-4932-b998-91bc4fce5ea3 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:41.938798 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1a69a-0806-4932-b998-91bc4fce5ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:41.939073 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1a69a-0806-4932-b998-91bc4fce5ea3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:41.939744 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d1a69a-0806-4932-b998-91bc4fce5ea3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:41.941085 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d1a69a-0806-4932-b998-91bc4fce5ea3 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:41.966072 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 467/931] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:15:41 2022] POST /identity/v3/endpoints => generated 347 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:15:42.669604 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3f455-0940-4b46-8c61-1a78a082ac1a None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:42.669604 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3f455-0940-4b46-8c61-1a78a082ac1a None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:42.669604 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3f455-0940-4b46-8c61-1a78a082ac1a None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:42.670333 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 465/932] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:42.675736 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b086d-1b0b-4cd2-a16e-6edab1a664eb None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:42.676039 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b086d-1b0b-4cd2-a16e-6edab1a664eb None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:42.676323 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b086d-1b0b-4cd2-a16e-6edab1a664eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:42.695424 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dadd09b-3b96-4b04-9e10-dadb0baae639 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:42.695739 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dadd09b-3b96-4b04-9e10-dadb0baae639 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:42.695965 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dadd09b-3b96-4b04-9e10-dadb0baae639 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:42.697587 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 466/933] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:15:42.699654 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-3f5b086d-1b0b-4cd2-a16e-6edab1a664eb None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:42.704852 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3f5b086d-1b0b-4cd2-a16e-6edab1a664eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:42.706365 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15f928-c652-4898-825c-30ed6b97989d None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:42.706866 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15f928-c652-4898-825c-30ed6b97989d None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:42.707339 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15f928-c652-4898-825c-30ed6b97989d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:42.728178 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-8b15f928-c652-4898-825c-30ed6b97989d None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:42.729505 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-8b15f928-c652-4898-825c-30ed6b97989d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:42.771868 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:42.797637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 468/934] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:15:42.813792 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 467/935] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:15:43.078531 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b086d-1b0b-4cd2-a16e-6edab1a664eb None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:43.093148 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b15f928-c652-4898-825c-30ed6b97989d None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:43.206560 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b15f928-c652-4898-825c-30ed6b97989d None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:43.211864 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b086d-1b0b-4cd2-a16e-6edab1a664eb None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:43.241364 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:43.250996 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3ed8bb-b3f0-4b02-bda5-01ad0377acd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:43.251601 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ed8bb-b3f0-4b02-bda5-01ad0377acd9 None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:43.251874 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ed8bb-b3f0-4b02-bda5-01ad0377acd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:43.252204 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ed8bb-b3f0-4b02-bda5-01ad0377acd9 None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:43.252849 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3ed8bb-b3f0-4b02-bda5-01ad0377acd9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:43.254572 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3ed8bb-b3f0-4b02-bda5-01ad0377acd9 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:43.257740 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: WARNING keystone.server.flask.common [None req-2d3ed8bb-b3f0-4b02-bda5-01ad0377acd9 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 09 20:15:43.262826 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3842d-f541-4e9b-bd2a-77d57056c4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:43.263710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3842d-f541-4e9b-bd2a-77d57056c4a5 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:43.264211 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3842d-f541-4e9b-bd2a-77d57056c4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:43.264702 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3842d-f541-4e9b-bd2a-77d57056c4a5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:43.276551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.sql.core [None req-2d3ed8bb-b3f0-4b02-bda5-01ad0377acd9 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 20:15:43.276551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 09 20:15:43.276551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [parameters: {'id': 'e1cd2a25982c4ff480ab2cb2b6b5b8cb', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 09 20:15:43.276551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81430) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 20:15:43.277525 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd3842d-f541-4e9b-bd2a-77d57056c4a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:15:43.282763 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-dfd3842d-f541-4e9b-bd2a-77d57056c4a5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 20:15:43.282763 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:43.282763 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:43.282763 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:43.282763 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:43.282763 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:43.282763 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:43.282763 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:43.282763 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:43.282763 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:43.282763 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:43.282763 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:43.282763 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:15:43.282763 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:15:43.282763 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:15:43.282763 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:15:43.283710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:15:43.283710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:15:43.283710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:15:43.283710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:15:43.283710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:43.283710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:43.283710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:43.283710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:43.283710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:43.283710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:43.283710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:43.283710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:43.283710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:43.283710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:43.283710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:43.283710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:43.284655 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:43.284655 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:43.284655 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:43.284655 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:43.284655 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:15:43.284655 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:15:43.284655 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:15:43.284655 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:15:43.284655 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:15:43.284655 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:15:43.284655 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 20:15:43.284655 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:43.285809 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-2d3ed8bb-b3f0-4b02-bda5-01ad0377acd9 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e1cd2a25982c4ff480ab2cb2b6b5b8cb.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e1cd2a25982c4ff480ab2cb2b6b5b8cb. Sep 09 20:15:43.285809 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:43.285809 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 20:15:43.285809 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 20:15:43.285809 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 20:15:43.285809 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 20:15:43.285809 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 20:15:43.285809 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 20:15:43.285809 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 20:15:43.285809 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application conn.query(q) Sep 09 20:15:43.285809 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 20:15:43.285809 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 20:15:43.285809 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 20:15:43.287131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application result.read() Sep 09 20:15:43.287131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 20:15:43.287131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 20:15:43.287131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 20:15:43.287131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 20:15:43.287131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 20:15:43.287131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 20:15:43.287131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 20:15:43.287131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 20:15:43.287131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 20:15:43.287131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:43.287131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 20:15:43.287131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:43.287131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:43.287131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 20:15:43.287131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 20:15:43.287131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 09 20:15:43.287131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 09 20:15:43.293562 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 20:15:43.293562 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application next(self.gen) Sep 09 20:15:43.293562 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 20:15:43.293562 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application yield resource Sep 09 20:15:43.293562 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 20:15:43.293562 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application next(self.gen) Sep 09 20:15:43.293562 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 20:15:43.293562 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 20:15:43.293562 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 20:15:43.293562 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 20:15:43.293562 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 20:15:43.293562 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise self.value Sep 09 20:15:43.293562 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 20:15:43.293562 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 20:15:43.293562 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 20:15:43.293562 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application session.commit() Sep 09 20:15:43.293562 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 20:15:43.295915 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 20:15:43.295915 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 20:15:43.295915 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 20:15:43.295915 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 20:15:43.295915 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self.session.flush() Sep 09 20:15:43.295915 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 20:15:43.295915 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 20:15:43.295915 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 20:15:43.295915 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 20:15:43.295915 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 20:15:43.295915 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application compat.raise_( Sep 09 20:15:43.295915 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 20:15:43.295915 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception Sep 09 20:15:43.295915 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 20:15:43.295915 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 20:15:43.295915 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 20:15:43.295915 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 20:15:43.298152 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 20:15:43.298152 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 20:15:43.298152 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 20:15:43.298152 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 20:15:43.298152 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 09 20:15:43.298152 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 09 20:15:43.298152 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 20:15:43.298152 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 20:15:43.298152 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 20:15:43.298152 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 20:15:43.298152 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 20:15:43.298152 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 20:15:43.298152 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 20:15:43.298152 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 20:15:43.298152 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 20:15:43.300328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 20:15:43.300328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 20:15:43.300328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception Sep 09 20:15:43.300328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 20:15:43.300328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 20:15:43.300328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 20:15:43.300328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 20:15:43.300328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 20:15:43.300328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 20:15:43.300328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 20:15:43.300328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application conn.query(q) Sep 09 20:15:43.300328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 20:15:43.300328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 20:15:43.300328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 20:15:43.300328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application result.read() Sep 09 20:15:43.300328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 20:15:43.300328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 20:15:43.303072 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 20:15:43.303072 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 20:15:43.303072 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 20:15:43.303072 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 20:15:43.303072 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 20:15:43.303072 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 20:15:43.303072 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 20:15:43.303072 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 09 20:15:43.303072 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [parameters: {'id': 'e1cd2a25982c4ff480ab2cb2b6b5b8cb', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 09 20:15:43.303072 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 20:15:43.303072 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:43.303072 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:15:43.303072 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:43.303072 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:43.303072 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 09 20:15:43.305105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 09 20:15:43.305105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 20:15:43.305105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 20:15:43.305105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 09 20:15:43.305105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:43.305105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:15:43.305105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:43.305105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:43.305105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:43.305105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:43.305105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:43.305105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:43.305105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:43.305105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:43.305105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:43.305105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:43.305105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:43.307350 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:43.307350 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 09 20:15:43.307350 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 09 20:15:43.307350 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:43.307350 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:43.307350 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 09 20:15:43.307350 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 09 20:15:43.307350 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e1cd2a25982c4ff480ab2cb2b6b5b8cb. Sep 09 20:15:43.307350 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:43.307350 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 468/936] 10.222.0.26 () {60 vars in 1140 bytes} [Fri Sep 9 20:15:43 2022] POST /identity/v3/projects => generated 262 bytes in 195 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:15:43.310746 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 469/937] 10.222.0.26 () {56 vars in 1120 bytes} [Fri Sep 9 20:15:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:43.310939 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3ed8bb-b3f0-4b02-bda5-01ad0377acd9 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:43.311091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3842d-f541-4e9b-bd2a-77d57056c4a5 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:43.360046 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:43.423657 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3842d-f541-4e9b-bd2a-77d57056c4a5 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:43.448057 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3ed8bb-b3f0-4b02-bda5-01ad0377acd9 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:43.463753 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af4236-db2b-411d-bb3f-1f22c2fe8503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:43.464714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af4236-db2b-411d-bb3f-1f22c2fe8503 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:43.465003 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af4236-db2b-411d-bb3f-1f22c2fe8503 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:43.465306 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af4236-db2b-411d-bb3f-1f22c2fe8503 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:43.472867 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af4236-db2b-411d-bb3f-1f22c2fe8503 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:43.475564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af4236-db2b-411d-bb3f-1f22c2fe8503 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:43.488491 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-c4af4236-db2b-411d-bb3f-1f22c2fe8503 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:43.488491 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:43.488491 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:43.488491 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:43.488491 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:43.488491 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:43.488491 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:43.488491 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:43.488491 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:43.488491 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:43.488491 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:43.488491 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:43.488491 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:15:43.488491 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:15:43.488491 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:15:43.490068 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:15:43.490068 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:43.490068 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:43.490068 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:43.490068 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:43.490068 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:43.490068 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:43.490068 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:43.490068 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:43.490068 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:43.490068 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:43.490068 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:43.490068 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:43.490068 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:43.490068 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:43.490068 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:43.490068 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:43.491004 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:15:43.491004 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:15:43.491004 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:15:43.491004 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:15:43.491004 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:15:43.491004 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:15:43.491004 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:43.491004 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:43.495188 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 470/938] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:15:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:43.496027 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af4236-db2b-411d-bb3f-1f22c2fe8503 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:43.499138 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90a7c3a-9119-4d48-9772-cc389feb5abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:43.500933 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a7c3a-9119-4d48-9772-cc389feb5abe None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:43.501461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a7c3a-9119-4d48-9772-cc389feb5abe None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:43.501943 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a7c3a-9119-4d48-9772-cc389feb5abe None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:43.502841 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90a7c3a-9119-4d48-9772-cc389feb5abe None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:43.503346 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90a7c3a-9119-4d48-9772-cc389feb5abe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:43.505689 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90a7c3a-9119-4d48-9772-cc389feb5abe None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:43.523168 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 469/939] 10.222.0.26 () {56 vars in 1115 bytes} [Fri Sep 9 20:15:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:43.529390 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90a7c3a-9119-4d48-9772-cc389feb5abe None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:43.591281 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:43.627496 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af4236-db2b-411d-bb3f-1f22c2fe8503 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:43.636318 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:43.651593 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90a7c3a-9119-4d48-9772-cc389feb5abe None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:43.669991 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bf2399-6624-4d33-bde0-b4d6e64adb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:43.670939 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf2399-6624-4d33-bde0-b4d6e64adb5d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:43.671471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf2399-6624-4d33-bde0-b4d6e64adb5d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:43.671950 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf2399-6624-4d33-bde0-b4d6e64adb5d None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:43.672818 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bf2399-6624-4d33-bde0-b4d6e64adb5d None admin] RBAC: Adding query filter params (name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:43.673349 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bf2399-6624-4d33-bde0-b4d6e64adb5d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:43.675467 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bf2399-6624-4d33-bde0-b4d6e64adb5d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:43.695855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 471/940] 10.222.0.26 () {56 vars in 1112 bytes} [Fri Sep 9 20:15:43 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:43.697821 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ffffc-893c-43fc-8560-8dc2f44655cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:43.699089 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ffffc-893c-43fc-8560-8dc2f44655cd None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:43.699614 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ffffc-893c-43fc-8560-8dc2f44655cd None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:43.700104 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ffffc-893c-43fc-8560-8dc2f44655cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:43.714737 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338ffffc-893c-43fc-8560-8dc2f44655cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:43.716651 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338ffffc-893c-43fc-8560-8dc2f44655cd None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:43.734216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-338ffffc-893c-43fc-8560-8dc2f44655cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:43.734216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:43.734216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:43.734216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:43.734216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:43.734216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:43.734216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:43.734216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:43.734216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:43.734216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:43.734216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:43.734216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:43.734216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:43.734216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:43.734216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:43.734216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:43.735273 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:43.735273 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:43.735273 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:43.735273 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:43.735273 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:43.735273 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:43.735273 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:43.735273 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:43.735273 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:43.735273 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:43.735273 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:43.735273 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:43.735273 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:43.735273 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:43.735273 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:43.735273 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:43.735273 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:43.736212 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:43.736212 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:43.736212 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:43.738534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 470/941] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:43.744894 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bf2399-6624-4d33-bde0-b4d6e64adb5d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:43.867493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bf2399-6624-4d33-bde0-b4d6e64adb5d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:43.906685 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676e3ea-ee7f-4d1f-aee7-ba3117e5a176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:43.907243 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676e3ea-ee7f-4d1f-aee7-ba3117e5a176 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:43.907527 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676e3ea-ee7f-4d1f-aee7-ba3117e5a176 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:43.907804 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676e3ea-ee7f-4d1f-aee7-ba3117e5a176 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:43.908460 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c676e3ea-ee7f-4d1f-aee7-ba3117e5a176 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:43.908765 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c676e3ea-ee7f-4d1f-aee7-ba3117e5a176 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:43.910154 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c676e3ea-ee7f-4d1f-aee7-ba3117e5a176 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:43.923935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 472/942] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:43.930252 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ffffc-893c-43fc-8560-8dc2f44655cd None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:43.947836 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4cdb4f-3309-40d9-be01-fd3d8ba99637 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:43.948168 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4cdb4f-3309-40d9-be01-fd3d8ba99637 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:43.948382 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4cdb4f-3309-40d9-be01-fd3d8ba99637 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:43.950315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 473/943] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:43.956327 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc452692-c072-44d9-8bcd-001c9c4a019f None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:43.956988 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc452692-c072-44d9-8bcd-001c9c4a019f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:43.958588 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc452692-c072-44d9-8bcd-001c9c4a019f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:43.982196 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-dc452692-c072-44d9-8bcd-001c9c4a019f None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:43.983842 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-dc452692-c072-44d9-8bcd-001c9c4a019f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:44.012675 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:44.057469 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:44.062791 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ffffc-893c-43fc-8560-8dc2f44655cd None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:44.077351 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 474/944] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:15:44.107075 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7336a4-d359-4d65-a0a6-7b3309e22933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:44.107784 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7336a4-d359-4d65-a0a6-7b3309e22933 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:44.108064 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7336a4-d359-4d65-a0a6-7b3309e22933 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:44.108336 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7336a4-d359-4d65-a0a6-7b3309e22933 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:44.116892 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7336a4-d359-4d65-a0a6-7b3309e22933 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:44.118418 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7336a4-d359-4d65-a0a6-7b3309e22933 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:44.130924 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-4a7336a4-d359-4d65-a0a6-7b3309e22933 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 20:15:44.130924 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:44.130924 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:44.130924 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:44.130924 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:44.130924 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:44.130924 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:44.130924 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:44.130924 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:44.130924 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:44.130924 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:44.130924 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:44.130924 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:15:44.130924 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:15:44.130924 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:15:44.130924 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:15:44.131892 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:44.131892 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:44.131892 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:15:44.131892 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:44.131892 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:15:44.131892 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:44.131892 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:44.131892 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:44.131892 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:44.131892 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:44.131892 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:44.131892 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:44.131892 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:44.131892 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:44.131892 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:44.131892 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:44.131892 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:44.131892 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:44.132847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:44.132847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:44.132847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:15:44.132847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:15:44.132847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:15:44.132847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:15:44.132847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:15:44.132847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:15:44.132847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 20:15:44.132847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:44.132847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 471/945] 10.222.0.26 () {56 vars in 1117 bytes} [Fri Sep 9 20:15:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:15:44.137745 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc452692-c072-44d9-8bcd-001c9c4a019f None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:44.197064 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3507ff-24b5-4d82-ad7f-5a1df60b1d2e None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:44.197383 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3507ff-24b5-4d82-ad7f-5a1df60b1d2e None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:44.197600 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3507ff-24b5-4d82-ad7f-5a1df60b1d2e None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:44.199682 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 472/946] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:44.205180 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9db6e-b302-444e-8ea3-39af24ba4c37 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:44.205498 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9db6e-b302-444e-8ea3-39af24ba4c37 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:44.205776 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9db6e-b302-444e-8ea3-39af24ba4c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:44.225576 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-f5d9db6e-b302-444e-8ea3-39af24ba4c37 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:44.227117 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-f5d9db6e-b302-444e-8ea3-39af24ba4c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:44.249990 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc452692-c072-44d9-8bcd-001c9c4a019f None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:44.293478 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad013571-984b-4c57-9854-a8f6aa7182e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:44.294465 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad013571-984b-4c57-9854-a8f6aa7182e6 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:44.295447 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad013571-984b-4c57-9854-a8f6aa7182e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:44.295936 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad013571-984b-4c57-9854-a8f6aa7182e6 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:44.297138 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad013571-984b-4c57-9854-a8f6aa7182e6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:44.297801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad013571-984b-4c57-9854-a8f6aa7182e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:44.299523 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad013571-984b-4c57-9854-a8f6aa7182e6 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:44.317821 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 473/947] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:15:44.320289 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 475/948] 10.222.0.26 () {56 vars in 1149 bytes} [Fri Sep 9 20:15:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:44.325692 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d9db6e-b302-444e-8ea3-39af24ba4c37 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:44.393613 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad013571-984b-4c57-9854-a8f6aa7182e6 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:44.410369 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:44.422097 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:44.435587 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d9db6e-b302-444e-8ea3-39af24ba4c37 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:44.474173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f5c86-9bf3-44f9-9bc8-f4d0863ebbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:44.474846 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f5c86-9bf3-44f9-9bc8-f4d0863ebbee None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:44.475121 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f5c86-9bf3-44f9-9bc8-f4d0863ebbee None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:44.475690 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f5c86-9bf3-44f9-9bc8-f4d0863ebbee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:44.488829 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8f5c86-9bf3-44f9-9bc8-f4d0863ebbee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:44.491572 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8f5c86-9bf3-44f9-9bc8-f4d0863ebbee None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:44.512503 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-cf8f5c86-9bf3-44f9-9bc8-f4d0863ebbee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:44.512503 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:44.512503 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:44.512503 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:44.512503 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:44.512503 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:44.512503 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:44.512503 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:44.512503 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:44.512503 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:44.512503 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:44.512503 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:44.512503 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:44.512503 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:44.512503 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:44.512503 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:44.513481 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:44.513481 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:44.513481 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:44.513481 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:44.513481 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:44.513481 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:44.513481 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:44.513481 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:44.513481 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:44.513481 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:44.513481 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:44.513481 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:44.513481 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:44.513481 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:44.513481 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:44.513481 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:44.513481 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:44.514395 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:44.514395 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:44.514395 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:44.514395 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 474/949] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:44.514621 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad013571-984b-4c57-9854-a8f6aa7182e6 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:44.519285 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f5c86-9bf3-44f9-9bc8-f4d0863ebbee None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:44.553048 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef13c0-569d-4169-9701-d5db847fdeba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:44.553849 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef13c0-569d-4169-9701-d5db847fdeba None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:44.554133 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef13c0-569d-4169-9701-d5db847fdeba None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:44.554411 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef13c0-569d-4169-9701-d5db847fdeba None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:44.555108 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef13c0-569d-4169-9701-d5db847fdeba None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:44.556460 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef13c0-569d-4169-9701-d5db847fdeba None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:44.566627 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-acef13c0-569d-4169-9701-d5db847fdeba None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 20:15:44.566627 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:44.566627 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:15:44.566627 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:15:44.566627 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:15:44.566627 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:15:44.566627 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:15:44.566627 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:44.566627 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 20:15:44.566627 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:44.566627 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:15:44.566627 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:44.566627 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:44.566627 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:44.566627 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:44.566627 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:44.566627 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:44.568202 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:44.568202 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:44.568202 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:44.568202 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:44.568202 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:44.568202 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:44.568202 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:15:44.568202 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:15:44.568202 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:15:44.568202 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:15:44.568202 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:44.568202 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:44.568202 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:44.568202 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:44.568202 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:44.568202 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:44.569091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:44.569091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:44.569091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:44.569091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:44.569091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:44.569091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:44.569091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:44.569091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:44.569091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:44.569091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:44.569091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:15:44.569091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:44.569091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 20:15:44.569091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:44.569091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 476/950] 10.222.0.26 () {56 vars in 1126 bytes} [Fri Sep 9 20:15:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:15:44.574331 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef13c0-569d-4169-9701-d5db847fdeba None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:44.637038 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f5c86-9bf3-44f9-9bc8-f4d0863ebbee None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:44.660483 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:44.683205 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b4661-403f-457c-bb69-c9d4362e5008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:44.683810 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b4661-403f-457c-bb69-c9d4362e5008 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:44.684102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b4661-403f-457c-bb69-c9d4362e5008 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:44.684348 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b4661-403f-457c-bb69-c9d4362e5008 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:44.684982 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b4661-403f-457c-bb69-c9d4362e5008 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:44.685627 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b4661-403f-457c-bb69-c9d4362e5008 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:44.686997 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b4661-403f-457c-bb69-c9d4362e5008 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:44.690545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef13c0-569d-4169-9701-d5db847fdeba None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:44.697201 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:44.705452 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 475/951] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:44.705452 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b4661-403f-457c-bb69-c9d4362e5008 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:44.731701 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f62180-14f9-4473-a8d8-9f259ef687b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:44.732513 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f62180-14f9-4473-a8d8-9f259ef687b1 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:44.732996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f62180-14f9-4473-a8d8-9f259ef687b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:44.733294 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f62180-14f9-4473-a8d8-9f259ef687b1 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:44.734120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f62180-14f9-4473-a8d8-9f259ef687b1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:44.734249 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f62180-14f9-4473-a8d8-9f259ef687b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:44.735559 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f62180-14f9-4473-a8d8-9f259ef687b1 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:44.745716 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 477/952] 10.222.0.26 () {56 vars in 1122 bytes} [Fri Sep 9 20:15:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:44.745716 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f62180-14f9-4473-a8d8-9f259ef687b1 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:44.811023 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b4661-403f-457c-bb69-c9d4362e5008 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:44.847381 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc78cd-de86-42c4-827b-36b509275c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:44.848008 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc78cd-de86-42c4-827b-36b509275c45 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:44.848284 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc78cd-de86-42c4-827b-36b509275c45 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:44.849080 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc78cd-de86-42c4-827b-36b509275c45 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:44.849680 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc78cd-de86-42c4-827b-36b509275c45 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:44.851226 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc78cd-de86-42c4-827b-36b509275c45 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:44.852132 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f62180-14f9-4473-a8d8-9f259ef687b1 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:44.859757 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-d0bc78cd-de86-42c4-827b-36b509275c45 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 20:15:44.859757 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:44.859757 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:15:44.859757 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:15:44.859757 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:15:44.859757 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:15:44.859757 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:15:44.859757 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:44.859757 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 20:15:44.859757 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:44.859757 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:15:44.859757 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:44.859757 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:44.859757 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:44.859757 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:44.859757 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:44.859757 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:44.860902 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:44.860902 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:44.860902 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:44.860902 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:44.860902 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:44.860902 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:44.860902 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:15:44.860902 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:15:44.860902 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:15:44.860902 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:15:44.860902 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:44.860902 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:44.860902 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:44.860902 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:44.860902 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:44.860902 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:44.861883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:44.861883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:44.861883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:44.861883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:44.861883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:44.861883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:44.861883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:44.861883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:44.861883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:44.861883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:44.861883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:15:44.861883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:44.861883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 20:15:44.861883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:44.865821 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 476/953] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:15:44 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:44.866426 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc78cd-de86-42c4-827b-36b509275c45 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:44.890831 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce4e6c0-ab50-40c6-8033-92aeb190cff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:44.892012 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4e6c0-ab50-40c6-8033-92aeb190cff6 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:44.892724 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4e6c0-ab50-40c6-8033-92aeb190cff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:44.893171 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4e6c0-ab50-40c6-8033-92aeb190cff6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:44.901455 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce4e6c0-ab50-40c6-8033-92aeb190cff6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:44.902365 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce4e6c0-ab50-40c6-8033-92aeb190cff6 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:44.913981 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-0ce4e6c0-ab50-40c6-8033-92aeb190cff6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:44.913981 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:44.913981 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:44.913981 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:44.913981 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:44.913981 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:44.913981 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:44.913981 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:44.913981 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:44.913981 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:44.913981 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:44.913981 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:44.913981 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:15:44.913981 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:15:44.913981 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:15:44.915074 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:15:44.915074 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:44.915074 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:44.915074 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:44.915074 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:44.915074 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:44.915074 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:44.915074 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:44.915074 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:44.915074 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:44.915074 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:44.915074 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:44.915074 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:44.915074 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:44.915074 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:44.915074 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:44.915074 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:44.916271 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:15:44.916271 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:15:44.916271 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:15:44.916271 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:15:44.916271 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:15:44.916271 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:15:44.916271 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:44.916271 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:44.916271 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 478/954] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:15:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:44.916271 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce4e6c0-ab50-40c6-8033-92aeb190cff6 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:44.918446 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:44.937329 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:44.975639 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc78cd-de86-42c4-827b-36b509275c45 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:45.017636 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20172ae-8ae5-4f14-85f9-4d6c39c3ed30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:45.018186 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20172ae-8ae5-4f14-85f9-4d6c39c3ed30 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:45.018464 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20172ae-8ae5-4f14-85f9-4d6c39c3ed30 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:45.018736 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20172ae-8ae5-4f14-85f9-4d6c39c3ed30 None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:45.019362 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20172ae-8ae5-4f14-85f9-4d6c39c3ed30 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:45.019665 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20172ae-8ae5-4f14-85f9-4d6c39c3ed30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:45.021094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20172ae-8ae5-4f14-85f9-4d6c39c3ed30 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:45.030945 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 477/955] 10.222.0.26 () {56 vars in 1122 bytes} [Fri Sep 9 20:15:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:45.031088 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce4e6c0-ab50-40c6-8033-92aeb190cff6 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:45.031172 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20172ae-8ae5-4f14-85f9-4d6c39c3ed30 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:45.070320 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b7a0a-86af-4c29-b0d6-8877c8750409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:45.071065 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b7a0a-86af-4c29-b0d6-8877c8750409 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:45.071454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b7a0a-86af-4c29-b0d6-8877c8750409 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:45.071822 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b7a0a-86af-4c29-b0d6-8877c8750409 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:45.072546 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8b7a0a-86af-4c29-b0d6-8877c8750409 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:45.072944 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8b7a0a-86af-4c29-b0d6-8877c8750409 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:45.074525 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8b7a0a-86af-4c29-b0d6-8877c8750409 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:45.084002 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 479/956] 10.222.0.26 () {56 vars in 1112 bytes} [Fri Sep 9 20:15:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:45.084002 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b7a0a-86af-4c29-b0d6-8877c8750409 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:45.137734 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20172ae-8ae5-4f14-85f9-4d6c39c3ed30 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:45.158532 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:45.164664 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:45.172899 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463571a7-2ea9-4ccd-97fd-44af9ab5c349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:45.173538 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463571a7-2ea9-4ccd-97fd-44af9ab5c349 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:45.173818 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463571a7-2ea9-4ccd-97fd-44af9ab5c349 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:45.174086 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463571a7-2ea9-4ccd-97fd-44af9ab5c349 None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:45.174773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463571a7-2ea9-4ccd-97fd-44af9ab5c349 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:45.175075 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463571a7-2ea9-4ccd-97fd-44af9ab5c349 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:45.176493 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463571a7-2ea9-4ccd-97fd-44af9ab5c349 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:45.188241 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 478/957] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:15:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:45.189044 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463571a7-2ea9-4ccd-97fd-44af9ab5c349 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:45.205503 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b7a0a-86af-4c29-b0d6-8877c8750409 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:45.244713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a813203f-1b6e-48a3-bd25-8779366ddfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:45.245835 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a813203f-1b6e-48a3-bd25-8779366ddfe6 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:45.246345 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a813203f-1b6e-48a3-bd25-8779366ddfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:45.246847 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a813203f-1b6e-48a3-bd25-8779366ddfe6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:45.247814 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a813203f-1b6e-48a3-bd25-8779366ddfe6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:45.249784 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a813203f-1b6e-48a3-bd25-8779366ddfe6 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:45.272316 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 480/958] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:15:45 2022] POST /identity/v3/endpoints => generated 349 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:15:45.273502 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a813203f-1b6e-48a3-bd25-8779366ddfe6 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:45.299479 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463571a7-2ea9-4ccd-97fd-44af9ab5c349 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:45.342996 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba6d6ed-84cb-4aa9-b205-0e97bc669e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:45.344652 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba6d6ed-84cb-4aa9-b205-0e97bc669e4d None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:45.344652 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba6d6ed-84cb-4aa9-b205-0e97bc669e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:45.344652 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba6d6ed-84cb-4aa9-b205-0e97bc669e4d None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:45.345481 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba6d6ed-84cb-4aa9-b205-0e97bc669e4d None admin] RBAC: Adding query filter params (type=compute) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:45.345949 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba6d6ed-84cb-4aa9-b205-0e97bc669e4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:45.347570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba6d6ed-84cb-4aa9-b205-0e97bc669e4d None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:45.359743 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 479/959] 10.222.0.26 () {56 vars in 1112 bytes} [Fri Sep 9 20:15:45 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:45.413420 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a813203f-1b6e-48a3-bd25-8779366ddfe6 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:45.445897 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:45.456784 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c99fc5-7fec-49de-9b82-e396bad86c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:45.458565 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c99fc5-7fec-49de-9b82-e396bad86c63 None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:45.459236 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c99fc5-7fec-49de-9b82-e396bad86c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:45.459626 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c99fc5-7fec-49de-9b82-e396bad86c63 None admin] PATH_INFO: `/v3/projects/933820e2adbc473bbd57a4d59be75d72/users/cedaf156c5584d62beedde598c94f322/roles/6f8d308c22cc406ba07771a051b4bde2` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:45.462573 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c99fc5-7fec-49de-9b82-e396bad86c63 None admin] RBAC: Authorizing `identity:create_grant(project_id=933820e2adbc473bbd57a4d59be75d72, user_id=cedaf156c5584d62beedde598c94f322, role_id=6f8d308c22cc406ba07771a051b4bde2)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:45.464273 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c99fc5-7fec-49de-9b82-e396bad86c63 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:45.482597 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 481/960] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Sep 9 20:15:45 2022] PUT /identity/v3/projects/933820e2adbc473bbd57a4d59be75d72/users/cedaf156c5584d62beedde598c94f322/roles/6f8d308c22cc406ba07771a051b4bde2 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:15:46.111972 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4e7cb-c653-4c01-8297-0d64bca56e74 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:46.112527 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4e7cb-c653-4c01-8297-0d64bca56e74 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:46.112527 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4e7cb-c653-4c01-8297-0d64bca56e74 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:46.114167 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 480/961] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:46.119807 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529d56b-6847-48f3-9a85-45d70e20ff04 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:46.120102 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529d56b-6847-48f3-9a85-45d70e20ff04 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:46.120385 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529d56b-6847-48f3-9a85-45d70e20ff04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:46.138808 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-b529d56b-6847-48f3-9a85-45d70e20ff04 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:46.140016 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b529d56b-6847-48f3-9a85-45d70e20ff04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:46.210583 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 482/962] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:15:46.487493 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4e7cb-c653-4c01-8297-0d64bca56e74 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:46.541173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:46.597660 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4e7cb-c653-4c01-8297-0d64bca56e74 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:46.632933 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a58a3-2b5c-42aa-91b7-a32d2a5238f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:46.633898 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a58a3-2b5c-42aa-91b7-a32d2a5238f6 None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:46.634394 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a58a3-2b5c-42aa-91b7-a32d2a5238f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:46.634853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a58a3-2b5c-42aa-91b7-a32d2a5238f6 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:46.635717 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177a58a3-2b5c-42aa-91b7-a32d2a5238f6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:46.637358 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177a58a3-2b5c-42aa-91b7-a32d2a5238f6 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:46.653057 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.sql.core [None req-177a58a3-2b5c-42aa-91b7-a32d2a5238f6 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 20:15:46.653057 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 09 20:15:46.653057 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [parameters: {'user_id': 'ba99df07738b4a038837d53b5beab33f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 09 20:15:46.653057 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81430) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 20:15:46.660616 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-177a58a3-2b5c-42aa-91b7-a32d2a5238f6 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 09 20:15:46.660616 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:46.660616 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 20:15:46.660616 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 20:15:46.660616 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 20:15:46.660616 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 20:15:46.660616 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 20:15:46.660616 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 20:15:46.660616 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 20:15:46.660616 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application conn.query(q) Sep 09 20:15:46.660616 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 20:15:46.660616 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 20:15:46.660616 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 20:15:46.660616 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application result.read() Sep 09 20:15:46.660616 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 20:15:46.661587 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 20:15:46.661587 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 20:15:46.661587 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 20:15:46.661587 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 20:15:46.661587 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 20:15:46.661587 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 20:15:46.661587 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 20:15:46.661587 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 20:15:46.661587 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:46.661587 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 20:15:46.661587 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:46.661587 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:46.661587 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 20:15:46.661587 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 20:15:46.661587 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 09 20:15:46.661587 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 09 20:15:46.661587 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 20:15:46.661587 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application next(self.gen) Sep 09 20:15:46.662579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 20:15:46.662579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application yield resource Sep 09 20:15:46.662579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 20:15:46.662579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application next(self.gen) Sep 09 20:15:46.662579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 20:15:46.662579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 20:15:46.662579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 20:15:46.662579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 20:15:46.662579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 20:15:46.662579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise self.value Sep 09 20:15:46.662579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 20:15:46.662579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 20:15:46.662579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 20:15:46.662579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application session.commit() Sep 09 20:15:46.662579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 20:15:46.662579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 20:15:46.662579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 20:15:46.663506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 20:15:46.663506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 20:15:46.663506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self.session.flush() Sep 09 20:15:46.663506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 20:15:46.663506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 20:15:46.663506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 20:15:46.663506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 20:15:46.663506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 20:15:46.663506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application compat.raise_( Sep 09 20:15:46.663506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 20:15:46.663506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception Sep 09 20:15:46.663506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 20:15:46.663506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 20:15:46.663506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 20:15:46.663506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 20:15:46.663506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 20:15:46.663506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 20:15:46.665159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 20:15:46.665159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 20:15:46.665159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 09 20:15:46.665159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 09 20:15:46.665159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 20:15:46.665159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 20:15:46.665159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 20:15:46.665159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 20:15:46.665159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 20:15:46.665159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 20:15:46.665159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 20:15:46.665159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 20:15:46.665159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 20:15:46.665159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 20:15:46.665159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 20:15:46.665159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception Sep 09 20:15:46.666087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 20:15:46.666087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 20:15:46.666087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 20:15:46.666087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 20:15:46.666087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 20:15:46.666087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 20:15:46.666087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 20:15:46.666087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application conn.query(q) Sep 09 20:15:46.666087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 20:15:46.666087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 20:15:46.666087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 20:15:46.666087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application result.read() Sep 09 20:15:46.666087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 20:15:46.666087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 20:15:46.666087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 20:15:46.666087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 20:15:46.667012 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 20:15:46.667012 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 20:15:46.667012 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 20:15:46.667012 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 20:15:46.667012 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 20:15:46.667012 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 09 20:15:46.667012 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ba99df07738b4a038837d53b5beab33f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 09 20:15:46.667012 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 20:15:46.667012 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:46.667012 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:15:46.667012 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:46.667012 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:46.667012 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:46.667012 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:46.667012 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:46.667862 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:46.667862 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:46.667862 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:46.667862 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:46.667862 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:46.667862 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:46.667862 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:46.667862 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 09 20:15:46.667862 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 09 20:15:46.667862 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:46.667862 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:46.667862 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:15:46.667862 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:46.667862 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:15:46.667862 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:46.667862 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 09 20:15:46.667862 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 09 20:15:46.669715 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 09 20:15:46.669715 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 09 20:15:46.669715 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 20:15:46.669715 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 20:15:46.669715 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 09 20:15:46.669715 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:46.672808 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 481/963] 10.222.0.26 () {60 vars in 1131 bytes} [Fri Sep 9 20:15:46 2022] POST /identity/v3/users => generated 160 bytes in 185 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:46.678166 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b529d56b-6847-48f3-9a85-45d70e20ff04 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:46.781286 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b529d56b-6847-48f3-9a85-45d70e20ff04 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:46.794754 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:46.819158 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b5779e-f6d1-4b50-835e-1de11024c927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:46.819802 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5779e-f6d1-4b50-835e-1de11024c927 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:46.820083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5779e-f6d1-4b50-835e-1de11024c927 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:46.820359 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5779e-f6d1-4b50-835e-1de11024c927 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:46.827565 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b5779e-f6d1-4b50-835e-1de11024c927 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:46.829056 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b5779e-f6d1-4b50-835e-1de11024c927 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:46.841040 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-16b5779e-f6d1-4b50-835e-1de11024c927 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 20:15:46.841040 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:46.841040 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:46.841040 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:46.841040 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:46.841040 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:46.841040 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:46.841040 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:46.841040 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:46.841040 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:46.841040 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:46.841040 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:46.841040 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:15:46.841040 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:15:46.841040 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:15:46.841040 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:15:46.842022 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:46.842022 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:46.842022 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:15:46.842022 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:46.842022 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:15:46.842022 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:46.842022 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:46.842022 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:46.842022 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:46.842022 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:46.842022 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:46.842022 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:46.842022 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:46.842022 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:46.842022 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:46.842022 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:46.842022 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:46.842022 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:46.842998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:46.842998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:46.842998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:15:46.842998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:15:46.842998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:15:46.842998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:15:46.842998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:15:46.842998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:15:46.842998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 20:15:46.842998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:46.847026 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 483/964] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:15:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:46.848645 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a58a3-2b5c-42aa-91b7-a32d2a5238f6 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:46.952664 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a58a3-2b5c-42aa-91b7-a32d2a5238f6 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:46.963754 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:46.992163 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc46385-f990-4aed-a0ba-6a48a7fc2eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:46.992771 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc46385-f990-4aed-a0ba-6a48a7fc2eff None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:46.993046 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc46385-f990-4aed-a0ba-6a48a7fc2eff None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:46.993342 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc46385-f990-4aed-a0ba-6a48a7fc2eff None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:46.994041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc46385-f990-4aed-a0ba-6a48a7fc2eff None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:46.994339 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc46385-f990-4aed-a0ba-6a48a7fc2eff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:46.995946 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc46385-f990-4aed-a0ba-6a48a7fc2eff None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:47.034895 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 482/965] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Sep 9 20:15:46 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:47.583195 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05aef1a-ed66-4672-a50c-b1d74353e440 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:47.583749 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05aef1a-ed66-4672-a50c-b1d74353e440 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:47.583982 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05aef1a-ed66-4672-a50c-b1d74353e440 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:47.585299 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 484/966] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:47.590500 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f932da-63ad-4916-8a8e-4ae269cc19b9 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:47.590926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f932da-63ad-4916-8a8e-4ae269cc19b9 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:47.591095 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f932da-63ad-4916-8a8e-4ae269cc19b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:47.609795 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-73f932da-63ad-4916-8a8e-4ae269cc19b9 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:47.611293 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-73f932da-63ad-4916-8a8e-4ae269cc19b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:47.646364 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400bddd8-86b0-41a8-ba86-046efa1616d3 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:47.646735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400bddd8-86b0-41a8-ba86-046efa1616d3 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:47.646997 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400bddd8-86b0-41a8-ba86-046efa1616d3 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:47.648435 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 485/967] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:15:47.654092 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea0615-7c6d-48c6-bb27-088991009d6f None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:47.654396 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea0615-7c6d-48c6-bb27-088991009d6f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:47.654657 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea0615-7c6d-48c6-bb27-088991009d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:47.677581 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e4ea0615-7c6d-48c6-bb27-088991009d6f None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:47.678836 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e4ea0615-7c6d-48c6-bb27-088991009d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:47.690256 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 483/968] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:15:47.753285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 486/969] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:15:47.753502 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66767afe-1426-4efc-abe5-72b911645a1a None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:47.753502 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66767afe-1426-4efc-abe5-72b911645a1a None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:47.753502 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66767afe-1426-4efc-abe5-72b911645a1a None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:47.754323 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 484/970] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:15:47.759880 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fc92e-da96-453b-ba04-0e00d040435d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:47.760166 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fc92e-da96-453b-ba04-0e00d040435d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:47.760438 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fc92e-da96-453b-ba04-0e00d040435d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:47.778656 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-df1fc92e-da96-453b-ba04-0e00d040435d None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:47.779852 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-df1fc92e-da96-453b-ba04-0e00d040435d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:47.861298 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 487/971] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:15:47.958716 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66767afe-1426-4efc-abe5-72b911645a1a None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:48.018921 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1fc92e-da96-453b-ba04-0e00d040435d None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:48.051397 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:48.062900 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:48.069132 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66767afe-1426-4efc-abe5-72b911645a1a None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:48.104252 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af9547-45cf-41e0-81e6-2341341c80d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:48.104856 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af9547-45cf-41e0-81e6-2341341c80d9 None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:48.105128 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af9547-45cf-41e0-81e6-2341341c80d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:48.105435 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af9547-45cf-41e0-81e6-2341341c80d9 None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:48.106125 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af9547-45cf-41e0-81e6-2341341c80d9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:48.107448 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af9547-45cf-41e0-81e6-2341341c80d9 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:48.122638 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 485/972] 10.222.0.26 () {60 vars in 1141 bytes} [Fri Sep 9 20:15:47 2022] POST /identity/v3/services => generated 242 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:15:48.123362 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af9547-45cf-41e0-81e6-2341341c80d9 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:48.125011 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1fc92e-da96-453b-ba04-0e00d040435d None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:48.173237 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f562dbb-ddb6-49d3-896d-cd75cd8c2a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:48.174214 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f562dbb-ddb6-49d3-896d-cd75cd8c2a75 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:48.174666 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f562dbb-ddb6-49d3-896d-cd75cd8c2a75 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:48.175140 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f562dbb-ddb6-49d3-896d-cd75cd8c2a75 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:48.175902 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f562dbb-ddb6-49d3-896d-cd75cd8c2a75 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:48.178299 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f562dbb-ddb6-49d3-896d-cd75cd8c2a75 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:48.188021 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-5f562dbb-ddb6-49d3-896d-cd75cd8c2a75 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 20:15:48.188021 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:48.188021 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:15:48.188021 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:15:48.188021 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:15:48.188021 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:15:48.188021 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:15:48.188021 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:48.188021 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 20:15:48.188021 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:48.188021 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:15:48.188021 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:48.188021 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:48.188021 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:48.188021 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:48.188021 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:48.188021 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:48.189156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:48.189156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:48.189156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:48.189156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:48.189156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:48.189156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:48.189156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:15:48.189156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:15:48.189156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:15:48.189156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:15:48.189156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:48.189156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:48.189156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:48.189156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:48.189156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:48.189156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:48.190081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:48.190081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:48.190081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:48.190081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:48.190081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:48.190081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:48.190081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:48.190081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:48.190081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:48.190081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:48.190081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:15:48.190081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:48.190081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 20:15:48.190081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:48.190081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 488/973] 10.222.0.26 () {56 vars in 1126 bytes} [Fri Sep 9 20:15:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:48.196001 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f562dbb-ddb6-49d3-896d-cd75cd8c2a75 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:48.245963 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af9547-45cf-41e0-81e6-2341341c80d9 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:48.284006 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475bdcc-a764-4743-a621-1f55addd131d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:48.284759 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475bdcc-a764-4743-a621-1f55addd131d None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:48.285113 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475bdcc-a764-4743-a621-1f55addd131d None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:48.285481 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475bdcc-a764-4743-a621-1f55addd131d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:48.299629 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9475bdcc-a764-4743-a621-1f55addd131d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:15:48.300333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:48.305399 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-9475bdcc-a764-4743-a621-1f55addd131d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 20:15:48.305399 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:48.305399 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:48.305399 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:48.305399 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:48.305399 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:48.305399 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:48.305399 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:48.305399 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:48.305399 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:48.305399 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:48.305399 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:48.305399 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:15:48.305399 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:15:48.305399 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:15:48.305399 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:15:48.306462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:15:48.306462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:15:48.306462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:15:48.306462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:15:48.306462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:48.306462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:48.306462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:48.306462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:48.306462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:48.306462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:48.306462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:48.306462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:48.306462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:48.306462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:48.306462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:48.306462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:48.307358 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:48.307358 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:48.307358 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:48.307358 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:48.307358 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:15:48.307358 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:15:48.307358 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:15:48.307358 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:15:48.307358 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:15:48.307358 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:15:48.307358 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 20:15:48.307358 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:48.308383 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 486/974] 10.222.0.26 () {56 vars in 1120 bytes} [Fri Sep 9 20:15:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:48.314828 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475bdcc-a764-4743-a621-1f55addd131d None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:48.316240 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f562dbb-ddb6-49d3-896d-cd75cd8c2a75 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:48.335687 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:48.357038 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6ddf8-2889-440c-ab4c-31dfaf740091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:48.357693 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6ddf8-2889-440c-ab4c-31dfaf740091 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:48.357974 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6ddf8-2889-440c-ab4c-31dfaf740091 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:48.358247 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6ddf8-2889-440c-ab4c-31dfaf740091 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:48.359301 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f6ddf8-2889-440c-ab4c-31dfaf740091 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:48.359611 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f6ddf8-2889-440c-ab4c-31dfaf740091 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:48.361353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f6ddf8-2889-440c-ab4c-31dfaf740091 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:48.371576 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 489/975] 10.222.0.26 () {56 vars in 1122 bytes} [Fri Sep 9 20:15:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:48.372575 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6ddf8-2889-440c-ab4c-31dfaf740091 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:48.429413 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475bdcc-a764-4743-a621-1f55addd131d None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:48.465389 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35a73b-25c1-4f8d-9758-cb024df17028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:48.466085 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35a73b-25c1-4f8d-9758-cb024df17028 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:48.466615 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35a73b-25c1-4f8d-9758-cb024df17028 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:48.466751 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35a73b-25c1-4f8d-9758-cb024df17028 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:48.467463 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35a73b-25c1-4f8d-9758-cb024df17028 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:48.467829 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35a73b-25c1-4f8d-9758-cb024df17028 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:48.469354 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35a73b-25c1-4f8d-9758-cb024df17028 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:48.478077 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 487/976] 10.222.0.26 () {56 vars in 1115 bytes} [Fri Sep 9 20:15:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:48.479345 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6ddf8-2889-440c-ab4c-31dfaf740091 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:48.483462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35a73b-25c1-4f8d-9758-cb024df17028 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:48.516630 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1274d-3e2c-4ffa-a967-41ab69a5db9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:48.517397 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1274d-3e2c-4ffa-a967-41ab69a5db9c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:48.517673 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1274d-3e2c-4ffa-a967-41ab69a5db9c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:48.517941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1274d-3e2c-4ffa-a967-41ab69a5db9c None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:48.518566 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac1274d-3e2c-4ffa-a967-41ab69a5db9c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:48.518862 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac1274d-3e2c-4ffa-a967-41ab69a5db9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:48.520230 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac1274d-3e2c-4ffa-a967-41ab69a5db9c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:48.526050 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 490/977] 10.222.0.26 () {56 vars in 1122 bytes} [Fri Sep 9 20:15:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:48.531832 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1274d-3e2c-4ffa-a967-41ab69a5db9c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:48.555298 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:48.570562 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:48.590629 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35a73b-25c1-4f8d-9758-cb024df17028 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:48.626787 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de318e9d-91ad-42bb-a3a4-c1c6aa927c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:48.627414 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de318e9d-91ad-42bb-a3a4-c1c6aa927c32 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:48.627690 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de318e9d-91ad-42bb-a3a4-c1c6aa927c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:48.627957 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de318e9d-91ad-42bb-a3a4-c1c6aa927c32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:48.638974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de318e9d-91ad-42bb-a3a4-c1c6aa927c32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:48.640737 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de318e9d-91ad-42bb-a3a4-c1c6aa927c32 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:48.647961 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1274d-3e2c-4ffa-a967-41ab69a5db9c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:48.656368 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-de318e9d-91ad-42bb-a3a4-c1c6aa927c32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:48.656368 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:48.656368 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:48.656368 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:48.656368 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:48.656368 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:48.656368 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:48.656368 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:48.656368 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:48.656368 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:48.656368 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:48.656368 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:48.656368 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:48.656368 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:48.656368 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:48.656368 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:48.657374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:48.657374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:48.657374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:48.657374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:48.657374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:48.657374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:48.657374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:48.657374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:48.657374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:48.657374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:48.657374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:48.657374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:48.657374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:48.657374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:48.657374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:48.657374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:48.657374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:48.663435 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:48.663435 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:48.663435 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:48.663435 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 488/978] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:15:48.663995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de318e9d-91ad-42bb-a3a4-c1c6aa927c32 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:48.696308 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca3fa2-95cc-4d1a-87ff-47ece291a6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:48.697166 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca3fa2-95cc-4d1a-87ff-47ece291a6e9 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:48.697492 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca3fa2-95cc-4d1a-87ff-47ece291a6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:48.697754 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca3fa2-95cc-4d1a-87ff-47ece291a6e9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:48.698463 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ca3fa2-95cc-4d1a-87ff-47ece291a6e9 None admin] RBAC: Adding query filter params (interface=admin, service_id=da7590d6ed8246e6aeeaa26461e5a88d, region_id=RegionOne) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:48.698770 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ca3fa2-95cc-4d1a-87ff-47ece291a6e9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:48.700121 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ca3fa2-95cc-4d1a-87ff-47ece291a6e9 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:48.711979 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 491/979] 10.222.0.26 () {56 vars in 1249 bytes} [Fri Sep 9 20:15:48 2022] GET /identity/v3/endpoints?service_id=da7590d6ed8246e6aeeaa26461e5a88d&interface=admin®ion_id=RegionOne => generated 193 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:48.713513 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca3fa2-95cc-4d1a-87ff-47ece291a6e9 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:48.799176 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de318e9d-91ad-42bb-a3a4-c1c6aa927c32 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:48.831580 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:48.834111 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:48.841260 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60f98d-1557-4ca4-a93e-a86319d1ec9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:48.841910 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60f98d-1557-4ca4-a93e-a86319d1ec9d None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:48.842162 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60f98d-1557-4ca4-a93e-a86319d1ec9d None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:48.842428 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60f98d-1557-4ca4-a93e-a86319d1ec9d None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:48.843067 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c60f98d-1557-4ca4-a93e-a86319d1ec9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:48.843359 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c60f98d-1557-4ca4-a93e-a86319d1ec9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:48.844720 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c60f98d-1557-4ca4-a93e-a86319d1ec9d None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:48.851459 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca3fa2-95cc-4d1a-87ff-47ece291a6e9 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:48.857113 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 489/980] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:48.862344 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60f98d-1557-4ca4-a93e-a86319d1ec9d None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:48.889675 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2671fa58-5d6c-4efe-b8f8-926f982b7fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:48.890509 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671fa58-5d6c-4efe-b8f8-926f982b7fcd None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:48.890941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671fa58-5d6c-4efe-b8f8-926f982b7fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:48.891222 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671fa58-5d6c-4efe-b8f8-926f982b7fcd None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:48.891823 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2671fa58-5d6c-4efe-b8f8-926f982b7fcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:48.893161 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2671fa58-5d6c-4efe-b8f8-926f982b7fcd None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:48.900125 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 492/981] 10.222.0.26 () {56 vars in 1087 bytes} [Fri Sep 9 20:15:48 2022] GET /identity/v3/services => generated 1727 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:15:48.970914 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60f98d-1557-4ca4-a93e-a86319d1ec9d None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:49.008886 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9dafb-d957-45a7-a0a5-d5eadd3be9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:49.009562 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9dafb-d957-45a7-a0a5-d5eadd3be9ef None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:49.009838 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9dafb-d957-45a7-a0a5-d5eadd3be9ef None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:49.010103 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9dafb-d957-45a7-a0a5-d5eadd3be9ef None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:49.018367 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9dafb-d957-45a7-a0a5-d5eadd3be9ef None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:49.019950 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9dafb-d957-45a7-a0a5-d5eadd3be9ef None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:49.034506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-00d9dafb-d957-45a7-a0a5-d5eadd3be9ef None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 20:15:49.034506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:49.034506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:49.034506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:49.034506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:49.034506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:49.034506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:49.034506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:49.034506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:49.034506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:49.034506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:49.034506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:49.034506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:15:49.034506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:15:49.034506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:15:49.034506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:15:49.035575 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:49.035575 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:49.035575 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:15:49.035575 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:49.035575 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:15:49.035575 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:49.035575 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:49.035575 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:49.035575 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:49.035575 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:49.035575 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:49.035575 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:49.035575 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:49.035575 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:49.035575 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:49.035575 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:49.035575 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:49.035575 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:49.036579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:49.036579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:49.036579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:15:49.036579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:15:49.036579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:15:49.036579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:15:49.036579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:15:49.036579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:15:49.036579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 20:15:49.036579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:49.038020 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 490/982] 10.222.0.26 () {56 vars in 1117 bytes} [Fri Sep 9 20:15:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:15:49.041982 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2671fa58-5d6c-4efe-b8f8-926f982b7fcd None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:49.146465 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2671fa58-5d6c-4efe-b8f8-926f982b7fcd None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:49.179092 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612e1b45-8943-4805-899f-32157daea3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:49.179695 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e1b45-8943-4805-899f-32157daea3a4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:49.179972 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e1b45-8943-4805-899f-32157daea3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:49.180241 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e1b45-8943-4805-899f-32157daea3a4 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:49.180961 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612e1b45-8943-4805-899f-32157daea3a4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:49.181278 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612e1b45-8943-4805-899f-32157daea3a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:49.182671 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612e1b45-8943-4805-899f-32157daea3a4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:49.200266 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 493/983] 10.222.0.26 () {56 vars in 1149 bytes} [Fri Sep 9 20:15:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:49.205671 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9dafb-d957-45a7-a0a5-d5eadd3be9ef None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:49.258898 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:49.307556 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9dafb-d957-45a7-a0a5-d5eadd3be9ef None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:49.340037 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc26abb-9305-4986-b64b-42616ac734c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:49.340635 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc26abb-9305-4986-b64b-42616ac734c2 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:49.340925 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc26abb-9305-4986-b64b-42616ac734c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:49.341182 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc26abb-9305-4986-b64b-42616ac734c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:49.409819 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc26abb-9305-4986-b64b-42616ac734c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:49.411944 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc26abb-9305-4986-b64b-42616ac734c2 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:49.461990 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:49.510615 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 491/984] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:15:49 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:15:49.515904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612e1b45-8943-4805-899f-32157daea3a4 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:49.525624 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:49.618915 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612e1b45-8943-4805-899f-32157daea3a4 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:49.657489 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e09032-815d-4e07-8881-b3f2b4a53a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:49.657489 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e09032-815d-4e07-8881-b3f2b4a53a6f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:49.657489 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e09032-815d-4e07-8881-b3f2b4a53a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:49.657489 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e09032-815d-4e07-8881-b3f2b4a53a6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:49.673454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e09032-815d-4e07-8881-b3f2b4a53a6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:49.675077 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e09032-815d-4e07-8881-b3f2b4a53a6f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:49.689234 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-69e09032-815d-4e07-8881-b3f2b4a53a6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:49.689234 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:49.689234 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:49.689234 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:49.689234 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:49.689234 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:49.689234 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:49.689234 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:49.689234 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:49.689234 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:49.689234 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:49.689234 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:49.689234 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:49.689234 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:49.689234 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:49.689234 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:49.690226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:49.690226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:49.690226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:49.690226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:49.690226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:49.690226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:49.690226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:49.690226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:49.690226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:49.690226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:49.690226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:49.690226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:49.690226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:49.690226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:49.690226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:49.690226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:49.690226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:49.691236 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:49.691236 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:49.691236 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:49.691236 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 494/985] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:49.697050 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc26abb-9305-4986-b64b-42616ac734c2 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:49.798290 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc26abb-9305-4986-b64b-42616ac734c2 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:49.834655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1f20a6-b2f6-457c-a213-c0f9e87fc9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:49.835267 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f20a6-b2f6-457c-a213-c0f9e87fc9f0 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:49.835545 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f20a6-b2f6-457c-a213-c0f9e87fc9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:49.835815 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f20a6-b2f6-457c-a213-c0f9e87fc9f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:49.836453 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1f20a6-b2f6-457c-a213-c0f9e87fc9f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:49.836751 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1f20a6-b2f6-457c-a213-c0f9e87fc9f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:49.838142 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1f20a6-b2f6-457c-a213-c0f9e87fc9f0 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:49.850285 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 492/986] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:49.855740 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e09032-815d-4e07-8881-b3f2b4a53a6f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:49.914464 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:49.956688 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e09032-815d-4e07-8881-b3f2b4a53a6f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:49.990378 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53fbd2-2e2c-4f8a-8c15-29740c96b526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:49.991006 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53fbd2-2e2c-4f8a-8c15-29740c96b526 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:49.991286 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53fbd2-2e2c-4f8a-8c15-29740c96b526 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:49.991561 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53fbd2-2e2c-4f8a-8c15-29740c96b526 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:49.998081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c53fbd2-2e2c-4f8a-8c15-29740c96b526 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:49.999499 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c53fbd2-2e2c-4f8a-8c15-29740c96b526 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:50.011337 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-0c53fbd2-2e2c-4f8a-8c15-29740c96b526 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:50.011337 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:50.011337 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:50.011337 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:50.011337 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:50.011337 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:50.011337 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:50.011337 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:50.011337 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:50.011337 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:50.011337 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:50.011337 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:50.011337 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:15:50.011337 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:15:50.011337 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:15:50.012223 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:15:50.012223 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:50.012223 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:50.012223 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:50.012223 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:50.012223 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:50.012223 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:50.012223 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:50.012223 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:50.012223 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:50.012223 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:50.012223 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:50.012223 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:50.012223 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:50.012223 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:50.012223 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:50.012223 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:50.013153 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:15:50.013153 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:15:50.013153 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:15:50.013153 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:15:50.013153 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:15:50.013153 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:15:50.013153 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:50.013153 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:50.017133 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 495/987] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:15:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:50.018554 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1f20a6-b2f6-457c-a213-c0f9e87fc9f0 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:50.035013 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:50.120049 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1f20a6-b2f6-457c-a213-c0f9e87fc9f0 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:50.154232 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9abe67-2d33-4c2f-81ad-1c95d5997474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:50.154842 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9abe67-2d33-4c2f-81ad-1c95d5997474 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:50.155121 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9abe67-2d33-4c2f-81ad-1c95d5997474 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:50.155394 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9abe67-2d33-4c2f-81ad-1c95d5997474 None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:50.156079 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9abe67-2d33-4c2f-81ad-1c95d5997474 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:50.156382 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9abe67-2d33-4c2f-81ad-1c95d5997474 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:50.157838 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9abe67-2d33-4c2f-81ad-1c95d5997474 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:50.170312 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 493/988] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:15:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:50.176195 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53fbd2-2e2c-4f8a-8c15-29740c96b526 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:50.282519 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53fbd2-2e2c-4f8a-8c15-29740c96b526 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:50.298908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:50.316135 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce0c7c-ce4d-4b7e-b0e4-eff323992de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:50.316738 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce0c7c-ce4d-4b7e-b0e4-eff323992de0 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:50.317572 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce0c7c-ce4d-4b7e-b0e4-eff323992de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:50.317846 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce0c7c-ce4d-4b7e-b0e4-eff323992de0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:50.318611 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce0c7c-ce4d-4b7e-b0e4-eff323992de0 None admin] RBAC: Adding query filter params (role.id=6f8d308c22cc406ba07771a051b4bde2, scope.project.id=933820e2adbc473bbd57a4d59be75d72, user.id=cedaf156c5584d62beedde598c94f322) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:50.318914 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce0c7c-ce4d-4b7e-b0e4-eff323992de0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:50.320326 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce0c7c-ce4d-4b7e-b0e4-eff323992de0 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:50.330981 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 496/989] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:15:50 2022] GET /identity/v3/role_assignments?user.id=cedaf156c5584d62beedde598c94f322&scope.project.id=933820e2adbc473bbd57a4d59be75d72&role.id=6f8d308c22cc406ba07771a051b4bde2 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:50.343984 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8bc75-5cd8-475a-853c-57d61e6eb5bc None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:50.344294 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8bc75-5cd8-475a-853c-57d61e6eb5bc None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:50.344526 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8bc75-5cd8-475a-853c-57d61e6eb5bc None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:50.345947 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 494/990] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:50.351976 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226af17c-1479-4d5f-a5b9-c9378665d78c None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:50.352268 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226af17c-1479-4d5f-a5b9-c9378665d78c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:50.352550 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226af17c-1479-4d5f-a5b9-c9378665d78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:50.371936 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-226af17c-1479-4d5f-a5b9-c9378665d78c None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:50.373077 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-226af17c-1479-4d5f-a5b9-c9378665d78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:50.451235 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 497/991] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:50 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:15:50.721412 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8bc75-5cd8-475a-853c-57d61e6eb5bc None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:50.795660 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:50.827838 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8bc75-5cd8-475a-853c-57d61e6eb5bc None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:50.865624 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be832f27-544b-4560-9846-8cca5f2c3b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:50.866630 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be832f27-544b-4560-9846-8cca5f2c3b92 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:50.867104 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be832f27-544b-4560-9846-8cca5f2c3b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:50.867590 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be832f27-544b-4560-9846-8cca5f2c3b92 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:50.868377 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be832f27-544b-4560-9846-8cca5f2c3b92 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:50.870122 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be832f27-544b-4560-9846-8cca5f2c3b92 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:50.880084 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-be832f27-544b-4560-9846-8cca5f2c3b92 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 20:15:50.880084 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:50.880084 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:15:50.880084 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:15:50.880084 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:15:50.880084 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:15:50.880084 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:15:50.880084 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:50.880084 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 20:15:50.880084 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:50.880084 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:15:50.880084 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:50.880084 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:50.880084 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:50.880084 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:50.880084 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:50.880084 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:50.881099 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:50.881099 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:50.881099 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:50.881099 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:50.881099 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:50.881099 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:50.881099 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:15:50.881099 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:15:50.881099 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:15:50.881099 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:15:50.881099 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:50.881099 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:50.881099 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:50.881099 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:50.881099 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:50.881099 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:50.882029 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:50.882029 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:50.882029 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:50.882029 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:50.882029 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:50.882029 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:50.882029 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:50.882029 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:50.882029 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:50.882029 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:50.882029 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:15:50.882029 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:50.882029 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 20:15:50.882029 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:50.883327 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 495/992] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:15:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:50.888864 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226af17c-1479-4d5f-a5b9-c9378665d78c None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:50.988740 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226af17c-1479-4d5f-a5b9-c9378665d78c None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:51.001928 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:51.025917 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0bc8fb-dcb4-4abe-9bbe-e48ab12de969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:51.026557 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bc8fb-dcb4-4abe-9bbe-e48ab12de969 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:51.026841 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bc8fb-dcb4-4abe-9bbe-e48ab12de969 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:51.027120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bc8fb-dcb4-4abe-9bbe-e48ab12de969 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:51.027765 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0bc8fb-dcb4-4abe-9bbe-e48ab12de969 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:51.028068 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0bc8fb-dcb4-4abe-9bbe-e48ab12de969 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:51.029506 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0bc8fb-dcb4-4abe-9bbe-e48ab12de969 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:51.040567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 498/993] 10.222.0.26 () {56 vars in 1112 bytes} [Fri Sep 9 20:15:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:51.041999 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be832f27-544b-4560-9846-8cca5f2c3b92 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:51.143140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be832f27-544b-4560-9846-8cca5f2c3b92 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:51.168137 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a3dd3-e448-4df7-bc73-463e5254c036 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:51.168425 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a3dd3-e448-4df7-bc73-463e5254c036 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:51.168684 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a3dd3-e448-4df7-bc73-463e5254c036 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:51.173400 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 499/994] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:15:51.175053 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aaead8-ec76-4de3-81ec-d690b26675c2 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:51.175342 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aaead8-ec76-4de3-81ec-d690b26675c2 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:51.175625 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aaead8-ec76-4de3-81ec-d690b26675c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:51.185417 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660953a-50ed-4fd1-9104-990d3fbfe35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:51.186026 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660953a-50ed-4fd1-9104-990d3fbfe35e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:51.186302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660953a-50ed-4fd1-9104-990d3fbfe35e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:51.186572 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660953a-50ed-4fd1-9104-990d3fbfe35e None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:51.187191 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7660953a-50ed-4fd1-9104-990d3fbfe35e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:51.187485 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7660953a-50ed-4fd1-9104-990d3fbfe35e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:51.189370 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7660953a-50ed-4fd1-9104-990d3fbfe35e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:51.191538 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:51.194803 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-02aaead8-ec76-4de3-81ec-d690b26675c2 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:51.195636 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 496/995] 10.222.0.26 () {56 vars in 1112 bytes} [Fri Sep 9 20:15:51 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:51.196198 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-02aaead8-ec76-4de3-81ec-d690b26675c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:51.201221 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660953a-50ed-4fd1-9104-990d3fbfe35e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:51.270871 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 500/996] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:51 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:15:51.316280 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660953a-50ed-4fd1-9104-990d3fbfe35e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:51.350758 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce939885-7deb-4f3f-b17e-d9e625d82684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:51.351354 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce939885-7deb-4f3f-b17e-d9e625d82684 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:51.351630 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce939885-7deb-4f3f-b17e-d9e625d82684 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:51.351895 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce939885-7deb-4f3f-b17e-d9e625d82684 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:51.352580 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce939885-7deb-4f3f-b17e-d9e625d82684 None admin] RBAC: Adding query filter params (interface=public, service_id=193c73fbbd164020aa3297fa295016b9, region_id=RegionOne) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:51.352880 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce939885-7deb-4f3f-b17e-d9e625d82684 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:51.354253 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce939885-7deb-4f3f-b17e-d9e625d82684 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:51.359361 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 497/997] 10.222.0.26 () {56 vars in 1251 bytes} [Fri Sep 9 20:15:51 2022] GET /identity/v3/endpoints?service_id=193c73fbbd164020aa3297fa295016b9&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:51.365389 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aaead8-ec76-4de3-81ec-d690b26675c2 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:51.469031 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aaead8-ec76-4de3-81ec-d690b26675c2 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:51.479447 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:51.509012 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73aa531-1e24-470b-99f0-af20dfb65f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:51.509654 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73aa531-1e24-470b-99f0-af20dfb65f6b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:51.509930 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73aa531-1e24-470b-99f0-af20dfb65f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:51.510197 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73aa531-1e24-470b-99f0-af20dfb65f6b None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:51.510846 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73aa531-1e24-470b-99f0-af20dfb65f6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:51.512193 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73aa531-1e24-470b-99f0-af20dfb65f6b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:51.525596 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 501/998] 10.222.0.26 () {56 vars in 1087 bytes} [Fri Sep 9 20:15:51 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:15:51.538163 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce939885-7deb-4f3f-b17e-d9e625d82684 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:51.608763 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:51.651634 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce939885-7deb-4f3f-b17e-d9e625d82684 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:51.699764 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32457375-a782-4987-bc81-f99596cfbf88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:51.700663 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32457375-a782-4987-bc81-f99596cfbf88 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:51.701111 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32457375-a782-4987-bc81-f99596cfbf88 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:51.701557 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32457375-a782-4987-bc81-f99596cfbf88 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:51.702311 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32457375-a782-4987-bc81-f99596cfbf88 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:51.703815 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32457375-a782-4987-bc81-f99596cfbf88 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:51.713216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-32457375-a782-4987-bc81-f99596cfbf88 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 20:15:51.713216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:51.713216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:15:51.713216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:15:51.713216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:15:51.713216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:15:51.713216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:15:51.713216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:51.713216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 20:15:51.713216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:51.713216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:15:51.713216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:51.713216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:51.713216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:51.713216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:51.713216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:51.713216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:51.714305 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:51.714305 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:51.714305 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:51.714305 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:51.714305 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:51.714305 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:51.714305 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:15:51.714305 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:15:51.714305 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:15:51.714305 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:15:51.714305 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:51.714305 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:51.714305 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:51.714305 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:51.714305 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:51.714305 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:51.715261 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:51.715261 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:51.715261 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:51.715261 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:51.715261 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:51.715261 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:51.715261 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:51.715261 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:51.715261 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:51.715261 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:51.715261 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:15:51.715261 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:51.715261 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 20:15:51.715261 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:51.718465 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 498/999] 10.222.0.26 () {56 vars in 1126 bytes} [Fri Sep 9 20:15:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:51.724614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73aa531-1e24-470b-99f0-af20dfb65f6b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:51.849222 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73aa531-1e24-470b-99f0-af20dfb65f6b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:51.884244 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c4f5a8-b3a7-4002-8ac7-c5fa04b5afab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:51.884907 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4f5a8-b3a7-4002-8ac7-c5fa04b5afab None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:51.885188 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4f5a8-b3a7-4002-8ac7-c5fa04b5afab None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:51.885490 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4f5a8-b3a7-4002-8ac7-c5fa04b5afab None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:51.886117 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c4f5a8-b3a7-4002-8ac7-c5fa04b5afab None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:51.886516 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c4f5a8-b3a7-4002-8ac7-c5fa04b5afab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:51.888461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c4f5a8-b3a7-4002-8ac7-c5fa04b5afab None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:51.898451 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 502/1000] 10.222.0.26 () {56 vars in 1122 bytes} [Fri Sep 9 20:15:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:51.899682 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32457375-a782-4987-bc81-f99596cfbf88 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:52.014455 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32457375-a782-4987-bc81-f99596cfbf88 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:52.046929 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:52.062496 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a4b6f4-b077-4a9a-97e3-65e26876dd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:52.063181 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a4b6f4-b077-4a9a-97e3-65e26876dd2b None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:52.063462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a4b6f4-b077-4a9a-97e3-65e26876dd2b None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:52.063736 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a4b6f4-b077-4a9a-97e3-65e26876dd2b None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:52.064420 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a4b6f4-b077-4a9a-97e3-65e26876dd2b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:52.064725 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a4b6f4-b077-4a9a-97e3-65e26876dd2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:52.066115 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a4b6f4-b077-4a9a-97e3-65e26876dd2b None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:52.076420 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 499/1001] 10.222.0.26 () {56 vars in 1122 bytes} [Fri Sep 9 20:15:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:52.077701 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c4f5a8-b3a7-4002-8ac7-c5fa04b5afab None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:52.116513 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:52.193676 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c4f5a8-b3a7-4002-8ac7-c5fa04b5afab None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:52.230947 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b41d6-113b-4b19-a121-94ee776f9fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:52.231566 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b41d6-113b-4b19-a121-94ee776f9fd7 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:52.231850 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b41d6-113b-4b19-a121-94ee776f9fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:52.232207 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b41d6-113b-4b19-a121-94ee776f9fd7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:52.232806 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84b41d6-113b-4b19-a121-94ee776f9fd7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:52.234192 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84b41d6-113b-4b19-a121-94ee776f9fd7 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:52.252705 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 503/1002] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:15:52 2022] POST /identity/v3/endpoints => generated 325 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:15:52.599368 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4be4e-5778-4e61-afb1-adc73cd653ef None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:52.599705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4be4e-5778-4e61-afb1-adc73cd653ef None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:52.599920 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4be4e-5778-4e61-afb1-adc73cd653ef None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:52.601414 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 500/1003] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:52.606419 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210b45b-d105-4b92-959b-9becd746b1f6 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:52.606704 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210b45b-d105-4b92-959b-9becd746b1f6 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:52.606979 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210b45b-d105-4b92-959b-9becd746b1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:52.625481 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-5210b45b-d105-4b92-959b-9becd746b1f6 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:52.626657 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5210b45b-d105-4b92-959b-9becd746b1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:52.705289 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 504/1004] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:52 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:15:52.964696 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4be4e-5778-4e61-afb1-adc73cd653ef None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:53.068635 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4be4e-5778-4e61-afb1-adc73cd653ef None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:53.101363 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e3bce-fba5-4814-8acc-df6a5b2ea03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:53.101981 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e3bce-fba5-4814-8acc-df6a5b2ea03e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:53.102257 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e3bce-fba5-4814-8acc-df6a5b2ea03e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:53.102523 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e3bce-fba5-4814-8acc-df6a5b2ea03e None admin] PATH_INFO: `/v3/services/image` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:53.103117 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e3bce-fba5-4814-8acc-df6a5b2ea03e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:53.104458 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e3bce-fba5-4814-8acc-df6a5b2ea03e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:53.112930 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-ba9e3bce-fba5-4814-8acc-df6a5b2ea03e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 20:15:53.112930 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:53.112930 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:15:53.112930 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:15:53.112930 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:15:53.112930 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:15:53.112930 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:15:53.112930 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:53.112930 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 20:15:53.112930 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:53.112930 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:15:53.112930 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:53.112930 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:53.112930 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:53.112930 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:53.112930 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:53.112930 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:53.114251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:53.114251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:53.114251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:53.114251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:53.114251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:53.114251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:53.114251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:15:53.114251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:15:53.114251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:15:53.114251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:15:53.114251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:53.114251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:53.114251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:53.114251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:53.114251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:53.114251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:53.115306 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:53.115306 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:53.115306 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:53.115306 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:53.115306 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:53.115306 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:53.115306 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:53.115306 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:53.115306 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:53.115306 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:53.115306 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:15:53.115306 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:53.115306 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 20:15:53.115306 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:53.117580 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 501/1005] 10.222.0.26 () {56 vars in 1105 bytes} [Fri Sep 9 20:15:52 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:53.119888 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5210b45b-d105-4b92-959b-9becd746b1f6 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:53.144404 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:53.226270 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5210b45b-d105-4b92-959b-9becd746b1f6 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:53.260026 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8853952f-15d7-4211-857c-4361d669debd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:53.260638 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853952f-15d7-4211-857c-4361d669debd None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:53.260921 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853952f-15d7-4211-857c-4361d669debd None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:53.261199 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853952f-15d7-4211-857c-4361d669debd None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:53.261859 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8853952f-15d7-4211-857c-4361d669debd None admin] RBAC: Adding query filter params (name=image) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:53.262164 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8853952f-15d7-4211-857c-4361d669debd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:53.263570 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8853952f-15d7-4211-857c-4361d669debd None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:53.273262 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 505/1006] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:53 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:53.274642 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e3bce-fba5-4814-8acc-df6a5b2ea03e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:53.333384 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:53.378919 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e3bce-fba5-4814-8acc-df6a5b2ea03e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:53.413228 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caca8e2f-e6d7-4865-ad39-11c4d1b3d2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:53.413851 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca8e2f-e6d7-4865-ad39-11c4d1b3d2d0 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:53.414178 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca8e2f-e6d7-4865-ad39-11c4d1b3d2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:53.414391 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca8e2f-e6d7-4865-ad39-11c4d1b3d2d0 None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:53.415014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caca8e2f-e6d7-4865-ad39-11c4d1b3d2d0 None admin] RBAC: Adding query filter params (type=image) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:53.415309 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caca8e2f-e6d7-4865-ad39-11c4d1b3d2d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:53.416661 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caca8e2f-e6d7-4865-ad39-11c4d1b3d2d0 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:53.421226 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 502/1007] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:53 2022] GET /identity/v3/services?type=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:53.776488 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4cb27a-0b1a-448c-9d08-6c0b83581c32 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:53.776832 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4cb27a-0b1a-448c-9d08-6c0b83581c32 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:53.777031 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4cb27a-0b1a-448c-9d08-6c0b83581c32 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:53.778524 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 506/1008] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:53.783502 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65959d97-4117-4e6f-a323-2e251a2e3b3a None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:53.783789 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65959d97-4117-4e6f-a323-2e251a2e3b3a None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:53.784066 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65959d97-4117-4e6f-a323-2e251a2e3b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:53.802121 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-65959d97-4117-4e6f-a323-2e251a2e3b3a None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:53.803303 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-65959d97-4117-4e6f-a323-2e251a2e3b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:53.873388 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 503/1009] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:53 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:15:54.140357 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4cb27a-0b1a-448c-9d08-6c0b83581c32 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:54.226863 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:54.244756 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4cb27a-0b1a-448c-9d08-6c0b83581c32 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:54.279077 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47067ff1-9ba3-4e53-a7f7-f18427a3dd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:54.279712 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47067ff1-9ba3-4e53-a7f7-f18427a3dd14 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:54.279984 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47067ff1-9ba3-4e53-a7f7-f18427a3dd14 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:54.280260 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47067ff1-9ba3-4e53-a7f7-f18427a3dd14 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:54.280835 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47067ff1-9ba3-4e53-a7f7-f18427a3dd14 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:54.282210 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47067ff1-9ba3-4e53-a7f7-f18427a3dd14 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:54.291817 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-47067ff1-9ba3-4e53-a7f7-f18427a3dd14 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 20:15:54.291817 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:54.291817 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:15:54.291817 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:15:54.291817 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:15:54.291817 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:15:54.291817 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:15:54.291817 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:54.291817 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 20:15:54.291817 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:54.291817 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:15:54.291817 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:54.291817 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:54.291817 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:54.291817 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:54.291817 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:54.291817 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:54.292940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:54.292940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:54.292940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:54.292940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:54.292940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:54.292940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:54.292940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:15:54.292940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:15:54.292940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:15:54.292940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:15:54.292940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:54.292940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:54.292940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:54.292940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:54.292940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:54.292940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:54.294288 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:54.294288 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:54.294288 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:54.294288 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:54.294288 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:54.294288 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:54.294288 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:54.294288 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:54.294288 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:54.294288 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:54.294288 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:15:54.294288 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:54.294288 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 20:15:54.294288 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:54.295559 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 507/1010] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:15:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:54.299484 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65959d97-4117-4e6f-a323-2e251a2e3b3a None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:54.354562 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:54.410020 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65959d97-4117-4e6f-a323-2e251a2e3b3a None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:54.419039 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44053801-2957-4139-a983-34d1a6165442 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:54.419324 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44053801-2957-4139-a983-34d1a6165442 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:54.419600 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44053801-2957-4139-a983-34d1a6165442 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:54.420920 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 508/1011] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:15:54.425880 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa1f95-d6b0-42a2-9069-feb4d913e4d4 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:54.426703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa1f95-d6b0-42a2-9069-feb4d913e4d4 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:54.426975 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa1f95-d6b0-42a2-9069-feb4d913e4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:54.445234 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-40fa1f95-d6b0-42a2-9069-feb4d913e4d4 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:54.446334 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f41c33-ab68-4eab-98da-3caf005b917a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:54.446460 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-40fa1f95-d6b0-42a2-9069-feb4d913e4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:54.446941 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f41c33-ab68-4eab-98da-3caf005b917a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:54.447217 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f41c33-ab68-4eab-98da-3caf005b917a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:54.447489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f41c33-ab68-4eab-98da-3caf005b917a None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:54.448154 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f41c33-ab68-4eab-98da-3caf005b917a None admin] RBAC: Adding query filter params (name=compute) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:54.448405 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f41c33-ab68-4eab-98da-3caf005b917a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:54.449803 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f41c33-ab68-4eab-98da-3caf005b917a None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:54.454631 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 504/1012] 10.222.0.26 () {56 vars in 1112 bytes} [Fri Sep 9 20:15:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:54.459970 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f41c33-ab68-4eab-98da-3caf005b917a None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:54.512692 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 509/1013] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:54 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:15:54.560462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f41c33-ab68-4eab-98da-3caf005b917a None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:54.574005 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:54.599448 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94277b6-4c8b-42c8-a4cd-b2681e83c27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:54.600054 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94277b6-4c8b-42c8-a4cd-b2681e83c27f None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:54.600334 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94277b6-4c8b-42c8-a4cd-b2681e83c27f None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:54.600605 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94277b6-4c8b-42c8-a4cd-b2681e83c27f None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:54.601224 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94277b6-4c8b-42c8-a4cd-b2681e83c27f None admin] RBAC: Adding query filter params (type=compute) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:54.601549 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94277b6-4c8b-42c8-a4cd-b2681e83c27f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:54.602897 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94277b6-4c8b-42c8-a4cd-b2681e83c27f None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:54.607811 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 505/1014] 10.222.0.26 () {56 vars in 1112 bytes} [Fri Sep 9 20:15:54 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:54.613323 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa1f95-d6b0-42a2-9069-feb4d913e4d4 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:54.706690 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:54.731985 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa1f95-d6b0-42a2-9069-feb4d913e4d4 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:54.765859 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66336b1-6e8c-4727-9883-bd6235d942d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:54.766514 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66336b1-6e8c-4727-9883-bd6235d942d8 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:54.766797 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66336b1-6e8c-4727-9883-bd6235d942d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:54.767070 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66336b1-6e8c-4727-9883-bd6235d942d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:54.767737 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66336b1-6e8c-4727-9883-bd6235d942d8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:54.769082 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66336b1-6e8c-4727-9883-bd6235d942d8 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:54.780452 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94277b6-4c8b-42c8-a4cd-b2681e83c27f None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:54.791623 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 510/1015] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:15:54 2022] POST /identity/v3/endpoints => generated 334 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:15:54.899580 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94277b6-4c8b-42c8-a4cd-b2681e83c27f None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:54.943756 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6efa9c8-6bd6-42c2-a5ce-77f4433fad2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:54.944382 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6efa9c8-6bd6-42c2-a5ce-77f4433fad2a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:54.944665 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6efa9c8-6bd6-42c2-a5ce-77f4433fad2a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:54.944936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6efa9c8-6bd6-42c2-a5ce-77f4433fad2a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:54.946180 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6efa9c8-6bd6-42c2-a5ce-77f4433fad2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:54.947690 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6efa9c8-6bd6-42c2-a5ce-77f4433fad2a None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:54.953284 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 506/1016] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:54.955350 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66336b1-6e8c-4727-9883-bd6235d942d8 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:55.064073 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66336b1-6e8c-4727-9883-bd6235d942d8 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:55.100526 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6dc35-9a65-43f7-8aff-4fc2acd8fb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:55.101408 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6dc35-9a65-43f7-8aff-4fc2acd8fb6b None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:55.101861 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6dc35-9a65-43f7-8aff-4fc2acd8fb6b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:55.102289 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6dc35-9a65-43f7-8aff-4fc2acd8fb6b None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:55.102943 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf6dc35-9a65-43f7-8aff-4fc2acd8fb6b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:55.104358 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf6dc35-9a65-43f7-8aff-4fc2acd8fb6b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:55.109313 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:55.135521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 511/1017] 10.222.0.26 () {60 vars in 1141 bytes} [Fri Sep 9 20:15:54 2022] POST /identity/v3/projects => generated 311 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:15:55.599278 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b20043-51e0-4e00-a770-6aabc7a8601b None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:55.599808 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b20043-51e0-4e00-a770-6aabc7a8601b None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:55.599808 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b20043-51e0-4e00-a770-6aabc7a8601b None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:55.601406 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 507/1018] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:55.606607 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd4b27-3373-48c6-ad9f-8ab78480005e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:55.607020 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd4b27-3373-48c6-ad9f-8ab78480005e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:55.607190 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd4b27-3373-48c6-ad9f-8ab78480005e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:55.625821 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-57cd4b27-3373-48c6-ad9f-8ab78480005e None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:55.626989 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-57cd4b27-3373-48c6-ad9f-8ab78480005e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:55.700869 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 512/1019] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:55 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:15:55.967222 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b20043-51e0-4e00-a770-6aabc7a8601b None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:56.013024 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:56.075809 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b20043-51e0-4e00-a770-6aabc7a8601b None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:56.109625 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160614c9-386e-4667-979b-1724b4aa8915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:56.110225 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160614c9-386e-4667-979b-1724b4aa8915 None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:56.110502 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160614c9-386e-4667-979b-1724b4aa8915 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:56.110770 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160614c9-386e-4667-979b-1724b4aa8915 None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:56.111416 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160614c9-386e-4667-979b-1724b4aa8915 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:56.112873 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160614c9-386e-4667-979b-1724b4aa8915 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:56.137273 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 508/1020] 10.222.0.26 () {60 vars in 1141 bytes} [Fri Sep 9 20:15:55 2022] POST /identity/v3/services => generated 242 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:15:57.064709 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3510f19-16b5-4493-b2f8-66b43a74dcb1 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:57.065441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3510f19-16b5-4493-b2f8-66b43a74dcb1 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:57.065441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3510f19-16b5-4493-b2f8-66b43a74dcb1 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:57.067257 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 513/1021] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:57.072171 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b305a-cab2-40cc-9e4b-9d930639329b None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:57.072559 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b305a-cab2-40cc-9e4b-9d930639329b None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:57.072755 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b305a-cab2-40cc-9e4b-9d930639329b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:57.091782 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-ab1b305a-cab2-40cc-9e4b-9d930639329b None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:57.092944 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-ab1b305a-cab2-40cc-9e4b-9d930639329b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:57.163957 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 509/1022] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:15:57.354031 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd99959-62a0-4386-98d0-dd227be856f7 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:57.354338 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd99959-62a0-4386-98d0-dd227be856f7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:57.354515 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd99959-62a0-4386-98d0-dd227be856f7 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:57.356011 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 514/1023] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:15:57.362037 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa39560-5990-45ac-960a-e5dc154b7ae8 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:57.362037 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa39560-5990-45ac-960a-e5dc154b7ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:57.362659 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa39560-5990-45ac-960a-e5dc154b7ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:57.383438 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-9fa39560-5990-45ac-960a-e5dc154b7ae8 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:57.384625 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-9fa39560-5990-45ac-960a-e5dc154b7ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:57.407433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:57.432428 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd99959-62a0-4386-98d0-dd227be856f7 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:57.455549 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 510/1024] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:15:57.532463 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd99959-62a0-4386-98d0-dd227be856f7 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:57.548968 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:57.568264 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de73c1ee-97d5-472c-9a06-5dcd34fa4e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:57.568888 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73c1ee-97d5-472c-9a06-5dcd34fa4e25 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:57.569162 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73c1ee-97d5-472c-9a06-5dcd34fa4e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:57.569459 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73c1ee-97d5-472c-9a06-5dcd34fa4e25 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:57.579299 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de73c1ee-97d5-472c-9a06-5dcd34fa4e25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:15:57.584023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-de73c1ee-97d5-472c-9a06-5dcd34fa4e25 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 20:15:57.584023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:57.584023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:57.584023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:57.584023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:57.584023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:57.584023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:57.584023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:57.584023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:57.584023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:57.584023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:57.584023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:57.584023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:15:57.584023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:15:57.584023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:15:57.584023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:15:57.584960 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:15:57.584960 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:15:57.584960 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:15:57.584960 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:15:57.584960 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:57.584960 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:57.584960 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:57.584960 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:57.584960 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:57.584960 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:57.584960 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:57.584960 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:57.584960 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:57.584960 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:57.584960 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:57.584960 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:57.585861 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:57.585861 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:57.585861 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:57.585861 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:57.585861 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:15:57.585861 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:15:57.585861 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:15:57.585861 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:15:57.585861 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:15:57.585861 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:15:57.585861 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 20:15:57.585861 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:57.590630 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 515/1025] 10.222.0.26 () {56 vars in 1120 bytes} [Fri Sep 9 20:15:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:15:57.590978 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa39560-5990-45ac-960a-e5dc154b7ae8 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:57.706607 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa39560-5990-45ac-960a-e5dc154b7ae8 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:57.740510 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de73c1ee-97d5-472c-9a06-5dcd34fa4e25 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:57.743533 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1cdab2-09a8-41a1-afa0-63d6ce260769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:57.744127 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1cdab2-09a8-41a1-afa0-63d6ce260769 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:57.744401 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1cdab2-09a8-41a1-afa0-63d6ce260769 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:57.744669 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1cdab2-09a8-41a1-afa0-63d6ce260769 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:57.745352 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1cdab2-09a8-41a1-afa0-63d6ce260769 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:57.745651 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1cdab2-09a8-41a1-afa0-63d6ce260769 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:57.747007 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1cdab2-09a8-41a1-afa0-63d6ce260769 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:57.755825 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 511/1026] 10.222.0.26 () {56 vars in 1115 bytes} [Fri Sep 9 20:15:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:57.761455 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1cdab2-09a8-41a1-afa0-63d6ce260769 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:57.790038 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:57.897168 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de73c1ee-97d5-472c-9a06-5dcd34fa4e25 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:57.905138 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1cdab2-09a8-41a1-afa0-63d6ce260769 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:57.939194 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08dc0b-18d4-48fe-84f0-bcf9b3d6f00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:57.940165 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08dc0b-18d4-48fe-84f0-bcf9b3d6f00a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:57.940654 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08dc0b-18d4-48fe-84f0-bcf9b3d6f00a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:57.941151 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08dc0b-18d4-48fe-84f0-bcf9b3d6f00a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:57.941747 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d874f95-e4c5-4485-9481-b5a4c9b532ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:57.942370 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d874f95-e4c5-4485-9481-b5a4c9b532ca None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:57.942644 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d874f95-e4c5-4485-9481-b5a4c9b532ca None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:57.942916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d874f95-e4c5-4485-9481-b5a4c9b532ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:57.943643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f08dc0b-18d4-48fe-84f0-bcf9b3d6f00a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:57.945831 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f08dc0b-18d4-48fe-84f0-bcf9b3d6f00a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:57.953103 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 516/1027] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:57.954736 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08dc0b-18d4-48fe-84f0-bcf9b3d6f00a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:57.956281 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d874f95-e4c5-4485-9481-b5a4c9b532ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:57.958291 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d874f95-e4c5-4485-9481-b5a4c9b532ca None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:57.977931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-3d874f95-e4c5-4485-9481-b5a4c9b532ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:57.977931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:57.977931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:57.977931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:57.977931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:57.977931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:57.977931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:57.977931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:57.977931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:57.977931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:57.977931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:57.977931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:57.977931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:57.977931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:57.977931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:57.977931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:57.978974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:57.978974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:57.978974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:57.978974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:57.978974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:57.978974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:57.978974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:57.978974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:57.978974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:57.978974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:57.978974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:57.978974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:57.978974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:57.978974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:57.978974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:57.978974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:57.978974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:57.979910 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:57.979910 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:57.979910 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:57.983901 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 512/1028] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:15:57.984767 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d874f95-e4c5-4485-9481-b5a4c9b532ca None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:58.016177 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:58.081185 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08dc0b-18d4-48fe-84f0-bcf9b3d6f00a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:58.085851 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:58.124310 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07069075-52a5-4860-893e-397507ca9acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:58.124921 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07069075-52a5-4860-893e-397507ca9acf None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:58.125205 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07069075-52a5-4860-893e-397507ca9acf None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:58.125521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07069075-52a5-4860-893e-397507ca9acf None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:58.126175 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07069075-52a5-4860-893e-397507ca9acf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:58.129193 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07069075-52a5-4860-893e-397507ca9acf None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:58.148964 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d874f95-e4c5-4485-9481-b5a4c9b532ca None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:58.157284 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 517/1029] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Sep 9 20:15:57 2022] POST /identity/v3/users => generated 285 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:15:58.199709 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ba06f8-53bd-48bc-adb8-c03b779b4264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:58.200634 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba06f8-53bd-48bc-adb8-c03b779b4264 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:58.201165 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba06f8-53bd-48bc-adb8-c03b779b4264 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:58.201633 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba06f8-53bd-48bc-adb8-c03b779b4264 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:58.202512 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ba06f8-53bd-48bc-adb8-c03b779b4264 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:58.202967 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ba06f8-53bd-48bc-adb8-c03b779b4264 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:58.204569 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ba06f8-53bd-48bc-adb8-c03b779b4264 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:58.218701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 513/1030] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:58.224525 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07069075-52a5-4860-893e-397507ca9acf None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:58.292541 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce627c-a788-4e31-b909-0f485a79e844 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:58.292828 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce627c-a788-4e31-b909-0f485a79e844 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:58.293099 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce627c-a788-4e31-b909-0f485a79e844 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:58.294633 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 514/1031] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:15:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:15:58.301720 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71352ad-91a3-49bc-883d-a8862502f796 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:58.302047 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71352ad-91a3-49bc-883d-a8862502f796 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:58.302270 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71352ad-91a3-49bc-883d-a8862502f796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:58.327606 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-f71352ad-91a3-49bc-883d-a8862502f796 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:15:58.328838 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-f71352ad-91a3-49bc-883d-a8862502f796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:15:58.341933 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:58.345460 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07069075-52a5-4860-893e-397507ca9acf None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:58.388774 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed726d-3dfc-4848-b0ed-382a2c22bdd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:58.389441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed726d-3dfc-4848-b0ed-382a2c22bdd6 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:58.389722 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed726d-3dfc-4848-b0ed-382a2c22bdd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:58.389996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed726d-3dfc-4848-b0ed-382a2c22bdd6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:58.399666 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed726d-3dfc-4848-b0ed-382a2c22bdd6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:58.401137 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed726d-3dfc-4848-b0ed-382a2c22bdd6 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:58.404902 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 515/1032] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:15:58 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:15:58.413968 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-a8ed726d-3dfc-4848-b0ed-382a2c22bdd6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:15:58.413968 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:58.413968 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:58.413968 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:58.413968 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:58.413968 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:58.413968 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:58.413968 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:58.413968 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:58.413968 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:58.413968 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:58.413968 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:58.413968 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:15:58.413968 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:15:58.413968 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:15:58.413968 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:15:58.414963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:58.414963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:58.414963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:15:58.414963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:58.414963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:15:58.414963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:15:58.414963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:58.414963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:58.414963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:58.414963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:58.414963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:58.414963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:58.414963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:58.414963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:58.414963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:58.414963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:58.414963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:58.414963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:58.415963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:58.415963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:58.415963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:15:58.415963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:15:58.415963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:15:58.415963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:15:58.415963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:15:58.415963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:15:58.415963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:15:58.415963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:58.415963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 518/1033] 10.222.0.26 () {56 vars in 1093 bytes} [Fri Sep 9 20:15:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:15:58.420833 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71352ad-91a3-49bc-883d-a8862502f796 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:58.484829 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:58.526021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71352ad-91a3-49bc-883d-a8862502f796 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:58.560038 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac873f1-22e9-423f-a8bb-b578fc11faf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:58.560646 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac873f1-22e9-423f-a8bb-b578fc11faf2 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:58.560919 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac873f1-22e9-423f-a8bb-b578fc11faf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:58.561186 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac873f1-22e9-423f-a8bb-b578fc11faf2 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:58.561948 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac873f1-22e9-423f-a8bb-b578fc11faf2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:58.562248 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac873f1-22e9-423f-a8bb-b578fc11faf2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:58.563661 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac873f1-22e9-423f-a8bb-b578fc11faf2 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:58.581207 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 516/1034] 10.222.0.26 () {56 vars in 1133 bytes} [Fri Sep 9 20:15:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:15:58.587518 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed726d-3dfc-4848-b0ed-382a2c22bdd6 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:58.681211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac873f1-22e9-423f-a8bb-b578fc11faf2 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:58.705626 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed726d-3dfc-4848-b0ed-382a2c22bdd6 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:58.742244 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e00a1-b424-4026-9775-30287c45d3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:58.742841 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e00a1-b424-4026-9775-30287c45d3ad None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:58.743044 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e00a1-b424-4026-9775-30287c45d3ad None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:58.743342 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e00a1-b424-4026-9775-30287c45d3ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:58.773769 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:58.800993 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac873f1-22e9-423f-a8bb-b578fc11faf2 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:58.817397 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e00a1-b424-4026-9775-30287c45d3ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:58.818967 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e00a1-b424-4026-9775-30287c45d3ad None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:58.822521 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:58.837800 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277fd26-e61f-42e9-9c88-bc6ff83d4d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:58.838430 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277fd26-e61f-42e9-9c88-bc6ff83d4d95 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:58.838710 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277fd26-e61f-42e9-9c88-bc6ff83d4d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:58.838985 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277fd26-e61f-42e9-9c88-bc6ff83d4d95 None admin] PATH_INFO: `/v3/services/image` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:58.839563 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6277fd26-e61f-42e9-9c88-bc6ff83d4d95 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:58.840910 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6277fd26-e61f-42e9-9c88-bc6ff83d4d95 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:58.850355 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-6277fd26-e61f-42e9-9c88-bc6ff83d4d95 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 20:15:58.850355 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:58.850355 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:15:58.850355 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:15:58.850355 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:15:58.850355 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:15:58.850355 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:15:58.850355 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:58.850355 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 20:15:58.850355 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:58.850355 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:15:58.850355 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:58.850355 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:58.850355 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:58.850355 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:58.850355 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:58.850355 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:58.851370 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:58.851370 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:58.851370 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:58.851370 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:58.851370 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:58.851370 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:58.851370 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:15:58.851370 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:15:58.851370 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:15:58.851370 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:15:58.851370 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:58.851370 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:58.851370 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:58.851370 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:58.851370 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:58.851370 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:58.852277 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:58.852277 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:58.852277 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:58.852277 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:58.852277 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:58.852277 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:58.852277 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:58.852277 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:58.852277 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:58.852277 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:58.852277 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:15:58.852277 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:15:58.852277 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 20:15:58.852277 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:15:58.852277 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 517/1035] 10.222.0.26 () {56 vars in 1105 bytes} [Fri Sep 9 20:15:58 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:15:58.857286 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277fd26-e61f-42e9-9c88-bc6ff83d4d95 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:58.934371 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 519/1036] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:15:58 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:15:58.940037 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e00a1-b424-4026-9775-30287c45d3ad None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:58.971388 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277fd26-e61f-42e9-9c88-bc6ff83d4d95 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:59.000613 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:59.009583 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b0405-c41e-426a-b0b9-7c79930a2af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:59.010191 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b0405-c41e-426a-b0b9-7c79930a2af4 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:59.010464 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b0405-c41e-426a-b0b9-7c79930a2af4 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:59.010808 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b0405-c41e-426a-b0b9-7c79930a2af4 None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:59.011426 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b0405-c41e-426a-b0b9-7c79930a2af4 None admin] RBAC: Adding query filter params (name=image) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:59.011725 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b0405-c41e-426a-b0b9-7c79930a2af4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:59.013075 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b0405-c41e-426a-b0b9-7c79930a2af4 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:59.017986 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 518/1037] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:59.023519 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b0405-c41e-426a-b0b9-7c79930a2af4 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:59.052106 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e00a1-b424-4026-9775-30287c45d3ad None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:59.072987 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:59.090044 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8bb053-3ed7-4477-8da2-e97c58ad7ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:59.090761 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8bb053-3ed7-4477-8da2-e97c58ad7ae3 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:59.091256 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8bb053-3ed7-4477-8da2-e97c58ad7ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:59.091532 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8bb053-3ed7-4477-8da2-e97c58ad7ae3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:59.104650 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8bb053-3ed7-4477-8da2-e97c58ad7ae3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:59.106525 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8bb053-3ed7-4477-8da2-e97c58ad7ae3 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:59.127677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-ff8bb053-3ed7-4477-8da2-e97c58ad7ae3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:59.127677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:59.127677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:59.127677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:59.127677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:59.127677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:59.127677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:59.127677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:59.127677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:59.127677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:59.127677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:59.127677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:59.127677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:15:59.127677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:15:59.127677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:15:59.127677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:15:59.129000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:59.129000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:59.129000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:59.129000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:59.129000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:59.129000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:59.129000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:59.129000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:59.129000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:59.129000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:59.129000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:59.129000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:59.129000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:59.129000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:59.129000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:59.129000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:59.129000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:15:59.130815 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:15:59.130815 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:15:59.130815 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:59.130815 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 520/1038] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:15:59.135069 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8bb053-3ed7-4477-8da2-e97c58ad7ae3 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:59.137373 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b0405-c41e-426a-b0b9-7c79930a2af4 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:59.181652 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b1d22-354b-4f05-a4e2-0dc057cb932e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:59.182750 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b1d22-354b-4f05-a4e2-0dc057cb932e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:59.183188 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b1d22-354b-4f05-a4e2-0dc057cb932e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:59.183466 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b1d22-354b-4f05-a4e2-0dc057cb932e None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:59.184092 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b1d22-354b-4f05-a4e2-0dc057cb932e None admin] RBAC: Adding query filter params (type=image) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:59.184650 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b1d22-354b-4f05-a4e2-0dc057cb932e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:59.186501 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b1d22-354b-4f05-a4e2-0dc057cb932e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:59.191382 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 519/1039] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:15:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:59.197572 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b1d22-354b-4f05-a4e2-0dc057cb932e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:59.244498 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8bb053-3ed7-4477-8da2-e97c58ad7ae3 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:59.258385 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:59.281590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98da1b1e-5ca0-4aef-a262-bb4f94a1a705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:59.282198 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da1b1e-5ca0-4aef-a262-bb4f94a1a705 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:59.282485 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da1b1e-5ca0-4aef-a262-bb4f94a1a705 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:59.282761 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da1b1e-5ca0-4aef-a262-bb4f94a1a705 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:59.283396 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98da1b1e-5ca0-4aef-a262-bb4f94a1a705 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:59.283696 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98da1b1e-5ca0-4aef-a262-bb4f94a1a705 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:59.285044 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98da1b1e-5ca0-4aef-a262-bb4f94a1a705 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:59.298137 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 521/1040] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:59.304107 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98da1b1e-5ca0-4aef-a262-bb4f94a1a705 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:59.309561 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b1d22-354b-4f05-a4e2-0dc057cb932e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:59.323464 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:59.346731 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42074b4b-5371-4820-ac75-55bbb0a391ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:59.347341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42074b4b-5371-4820-ac75-55bbb0a391ee None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:59.347619 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42074b4b-5371-4820-ac75-55bbb0a391ee None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:59.347893 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42074b4b-5371-4820-ac75-55bbb0a391ee None admin] PATH_INFO: `/v3/endpoints` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:59.348593 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42074b4b-5371-4820-ac75-55bbb0a391ee None admin] RBAC: Adding query filter params (interface=public, service_id=ea26fc63fb2646e680ac582e279ea735, region_id=RegionOne) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:59.348902 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42074b4b-5371-4820-ac75-55bbb0a391ee None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:59.350336 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42074b4b-5371-4820-ac75-55bbb0a391ee None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:59.355396 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 520/1041] 10.222.0.26 () {56 vars in 1251 bytes} [Fri Sep 9 20:15:59 2022] GET /identity/v3/endpoints?service_id=ea26fc63fb2646e680ac582e279ea735&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:59.360890 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42074b4b-5371-4820-ac75-55bbb0a391ee None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:59.420487 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98da1b1e-5ca0-4aef-a262-bb4f94a1a705 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:59.460661 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb1590-edff-444f-9a58-3930bd847eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:59.461501 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb1590-edff-444f-9a58-3930bd847eba None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:59.462105 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb1590-edff-444f-9a58-3930bd847eba None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:59.462403 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb1590-edff-444f-9a58-3930bd847eba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:59.464603 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42074b4b-5371-4820-ac75-55bbb0a391ee None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:59.469750 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb1590-edff-444f-9a58-3930bd847eba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:59.471188 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb1590-edff-444f-9a58-3930bd847eba None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:59.483521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-94fb1590-edff-444f-9a58-3930bd847eba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:59.483521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:15:59.483521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:15:59.483521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:15:59.483521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:15:59.483521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:15:59.483521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:15:59.483521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:15:59.483521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:15:59.483521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:15:59.483521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:15:59.483521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:15:59.483521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:15:59.483521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:15:59.483521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:15:59.484482 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:15:59.484482 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:15:59.484482 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:15:59.484482 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:15:59.484482 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:15:59.484482 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:15:59.484482 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:15:59.484482 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:15:59.484482 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:15:59.484482 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:15:59.484482 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:15:59.484482 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:15:59.484482 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:15:59.484482 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:15:59.484482 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:15:59.484482 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:15:59.484482 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:15:59.485477 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:15:59.485477 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:15:59.485477 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:15:59.485477 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:15:59.485477 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:15:59.485477 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:15:59.485477 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:15:59.485477 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:15:59.485477 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 522/1042] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:15:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:15:59.490912 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb1590-edff-444f-9a58-3930bd847eba None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:59.503859 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1addc-f7dd-4f98-850b-f43697771bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:59.505325 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1addc-f7dd-4f98-850b-f43697771bb4 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:59.505611 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1addc-f7dd-4f98-850b-f43697771bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:59.505884 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1addc-f7dd-4f98-850b-f43697771bb4 None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:59.506759 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef1addc-f7dd-4f98-850b-f43697771bb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:59.508964 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef1addc-f7dd-4f98-850b-f43697771bb4 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:59.527512 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 521/1043] 10.222.0.26 () {56 vars in 1087 bytes} [Fri Sep 9 20:15:59 2022] GET /identity/v3/services => generated 1957 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:15:59.527741 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:59.608291 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb1590-edff-444f-9a58-3930bd847eba None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:59.645414 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccc40ca-4ea4-49d3-8382-f865727a4013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:59.646044 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc40ca-4ea4-49d3-8382-f865727a4013 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:59.646321 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc40ca-4ea4-49d3-8382-f865727a4013 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:59.646598 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc40ca-4ea4-49d3-8382-f865727a4013 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:59.647310 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccc40ca-4ea4-49d3-8382-f865727a4013 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:59.647611 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccc40ca-4ea4-49d3-8382-f865727a4013 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:59.649066 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccc40ca-4ea4-49d3-8382-f865727a4013 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:59.669729 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 523/1044] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:15:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:15:59.671208 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1addc-f7dd-4f98-850b-f43697771bb4 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:15:59.717953 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:15:59.782898 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1addc-f7dd-4f98-850b-f43697771bb4 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:15:59.819654 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baaf3be-0432-471d-aedb-afb576b8e42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:15:59.820263 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baaf3be-0432-471d-aedb-afb576b8e42b None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:15:59.820543 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baaf3be-0432-471d-aedb-afb576b8e42b None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:15:59.820817 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baaf3be-0432-471d-aedb-afb576b8e42b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:15:59.821570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baaf3be-0432-471d-aedb-afb576b8e42b None admin] RBAC: Adding query filter params (role.id=6f8d308c22cc406ba07771a051b4bde2, scope.project.id=933820e2adbc473bbd57a4d59be75d72, user.id=87ece6128f9245f7b49f3e7a9faaed22) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:15:59.821878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baaf3be-0432-471d-aedb-afb576b8e42b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:15:59.823298 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baaf3be-0432-471d-aedb-afb576b8e42b None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:15:59.828881 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 522/1045] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:15:59 2022] GET /identity/v3/role_assignments?user.id=87ece6128f9245f7b49f3e7a9faaed22&scope.project.id=933820e2adbc473bbd57a4d59be75d72&role.id=6f8d308c22cc406ba07771a051b4bde2 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:00.439410 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962aa15-8076-484b-a903-7e4208a3c31c None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:00.439970 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962aa15-8076-484b-a903-7e4208a3c31c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:00.439970 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962aa15-8076-484b-a903-7e4208a3c31c None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:00.441359 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 524/1046] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:16:00.447532 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225cb206-09e8-4941-a62d-489c400f48bb None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:00.447941 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225cb206-09e8-4941-a62d-489c400f48bb None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:00.448115 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225cb206-09e8-4941-a62d-489c400f48bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:00.466139 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-225cb206-09e8-4941-a62d-489c400f48bb None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:16:00.467306 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-225cb206-09e8-4941-a62d-489c400f48bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:16:00.535183 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 523/1047] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:16:00 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:16:00.807048 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962aa15-8076-484b-a903-7e4208a3c31c None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:00.900021 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:00.910834 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962aa15-8076-484b-a903-7e4208a3c31c None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:00.944836 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdcb084-8a74-46fd-a9ba-1658719f2cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:00.945488 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcb084-8a74-46fd-a9ba-1658719f2cee None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:00.945769 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcb084-8a74-46fd-a9ba-1658719f2cee None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:00.946041 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcb084-8a74-46fd-a9ba-1658719f2cee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:00.955866 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdcb084-8a74-46fd-a9ba-1658719f2cee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:16:00.960674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-7cdcb084-8a74-46fd-a9ba-1658719f2cee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:16:00.960674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:16:00.960674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:16:00.960674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:16:00.960674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:16:00.960674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:16:00.960674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:16:00.960674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:16:00.960674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:16:00.960674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:16:00.960674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:16:00.960674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:16:00.960674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:16:00.960674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:16:00.960674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:16:00.960674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:16:00.961647 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:16:00.961647 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:16:00.961647 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:16:00.961647 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:16:00.961647 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:16:00.961647 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:16:00.961647 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:16:00.961647 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:16:00.961647 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:16:00.961647 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:16:00.961647 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:16:00.961647 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:16:00.961647 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:16:00.961647 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:16:00.961647 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:16:00.961647 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:16:00.962527 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:16:00.962527 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:16:00.962527 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:16:00.962527 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:16:00.962527 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:16:00.962527 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:16:00.962527 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:16:00.962527 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:16:00.962527 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:16:00.962527 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:16:00.962527 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:16:00.962527 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:16:00.965560 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 525/1048] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:16:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:16:00.967871 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225cb206-09e8-4941-a62d-489c400f48bb None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:01.002459 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:01.072871 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225cb206-09e8-4941-a62d-489c400f48bb None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:01.106444 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d4d2fc-9285-4b6c-aff1-f983a3dc50d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:01.107054 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4d2fc-9285-4b6c-aff1-f983a3dc50d5 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:01.107333 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4d2fc-9285-4b6c-aff1-f983a3dc50d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:01.107604 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4d2fc-9285-4b6c-aff1-f983a3dc50d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:01.108269 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d4d2fc-9285-4b6c-aff1-f983a3dc50d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:16:01.108573 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d4d2fc-9285-4b6c-aff1-f983a3dc50d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:01.109958 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d4d2fc-9285-4b6c-aff1-f983a3dc50d5 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:01.118996 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 524/1049] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:16:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:01.124693 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdcb084-8a74-46fd-a9ba-1658719f2cee None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:01.224017 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdcb084-8a74-46fd-a9ba-1658719f2cee None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:01.258688 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b076d4-449b-4638-9d21-5ff43ec1f095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:01.259321 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b076d4-449b-4638-9d21-5ff43ec1f095 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:01.259604 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b076d4-449b-4638-9d21-5ff43ec1f095 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:01.259882 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b076d4-449b-4638-9d21-5ff43ec1f095 None admin] PATH_INFO: `/v3/users/7adec72566154be7b81d03fa02949afb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:01.272856 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b076d4-449b-4638-9d21-5ff43ec1f095 None admin] RBAC: Authorizing `identity:get_user(user_id=7adec72566154be7b81d03fa02949afb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:01.274327 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b076d4-449b-4638-9d21-5ff43ec1f095 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:01.276163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 526/1050] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:16:01 2022] GET /identity/v3/users/7adec72566154be7b81d03fa02949afb => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:01.281697 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d4d2fc-9285-4b6c-aff1-f983a3dc50d5 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:01.378379 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:01.386163 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d4d2fc-9285-4b6c-aff1-f983a3dc50d5 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:01.419875 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7f533-251d-49b2-a07c-714d3397cff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:01.420479 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7f533-251d-49b2-a07c-714d3397cff4 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:01.420761 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7f533-251d-49b2-a07c-714d3397cff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:01.421032 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7f533-251d-49b2-a07c-714d3397cff4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:01.487200 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c7f533-251d-49b2-a07c-714d3397cff4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:01.489230 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c7f533-251d-49b2-a07c-714d3397cff4 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:01.586401 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:01.599048 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 525/1051] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:16:01 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 20:16:01.600949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b076d4-449b-4638-9d21-5ff43ec1f095 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:01.614538 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:01.674991 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12822de-26de-495e-8592-37299a317a4a None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:01.675273 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12822de-26de-495e-8592-37299a317a4a None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:01.675539 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12822de-26de-495e-8592-37299a317a4a None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:01.677056 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 526/1052] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:16:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:16:01.681760 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d696f-a748-4e2b-bc71-29a6137e2d3c None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:01.682040 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d696f-a748-4e2b-bc71-29a6137e2d3c None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:01.682314 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d696f-a748-4e2b-bc71-29a6137e2d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:01.699826 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-c76d696f-a748-4e2b-bc71-29a6137e2d3c None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:16:01.700992 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-c76d696f-a748-4e2b-bc71-29a6137e2d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:16:01.721987 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b076d4-449b-4638-9d21-5ff43ec1f095 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:01.759845 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d278a6-f046-455e-bb8b-4715357bb710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:01.760338 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d278a6-f046-455e-bb8b-4715357bb710 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:01.760621 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d278a6-f046-455e-bb8b-4715357bb710 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:01.760898 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d278a6-f046-455e-bb8b-4715357bb710 None admin] PATH_INFO: `/v3/projects/81d44cfe705d437987b23852c51e07cc` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:01.762150 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d278a6-f046-455e-bb8b-4715357bb710 None admin] RBAC: Authorizing `identity:get_project(project_id=81d44cfe705d437987b23852c51e07cc)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:01.763575 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d278a6-f046-455e-bb8b-4715357bb710 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:01.765214 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 527/1053] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:16:01 2022] GET /identity/v3/projects/81d44cfe705d437987b23852c51e07cc => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:01.772070 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d278a6-f046-455e-bb8b-4715357bb710 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:01.785285 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 527/1054] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:16:01 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:16:01.850682 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:01.875439 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d278a6-f046-455e-bb8b-4715357bb710 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:01.921032 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0d141-62e5-4d21-a938-e236edb3edd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:01.921664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0d141-62e5-4d21-a938-e236edb3edd8 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:01.921946 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0d141-62e5-4d21-a938-e236edb3edd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:01.922221 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0d141-62e5-4d21-a938-e236edb3edd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:01.922946 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb0d141-62e5-4d21-a938-e236edb3edd8 None admin] RBAC: Adding query filter params (role.id=9121ba25a6d84756990f0c3a19e9fecb, scope.project.id=81d44cfe705d437987b23852c51e07cc, user.id=7adec72566154be7b81d03fa02949afb) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:16:01.923255 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb0d141-62e5-4d21-a938-e236edb3edd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:01.924716 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb0d141-62e5-4d21-a938-e236edb3edd8 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:01.937563 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 528/1055] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:16:01 2022] GET /identity/v3/role_assignments?user.id=7adec72566154be7b81d03fa02949afb&scope.project.id=81d44cfe705d437987b23852c51e07cc&role.id=9121ba25a6d84756990f0c3a19e9fecb => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:16:01.983249 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618f731-8543-473c-a5f9-0ef533db008c None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:01.983529 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618f731-8543-473c-a5f9-0ef533db008c None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:01.983808 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618f731-8543-473c-a5f9-0ef533db008c None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:01.985699 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 528/1056] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:16:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:16:01.991368 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1885ee6-83dd-4497-afc3-d8b00d958856 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:01.991731 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1885ee6-83dd-4497-afc3-d8b00d958856 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:01.991940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1885ee6-83dd-4497-afc3-d8b00d958856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:02.011944 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-d1885ee6-83dd-4497-afc3-d8b00d958856 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:16:02.013182 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d1885ee6-83dd-4497-afc3-d8b00d958856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:16:02.042345 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618f731-8543-473c-a5f9-0ef533db008c None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:02.097295 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 529/1057] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:16:01 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:16:02.152550 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618f731-8543-473c-a5f9-0ef533db008c None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:02.179016 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:02.188543 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d178d87-a5cf-4ddc-a1d7-96484674db68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:02.189170 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d178d87-a5cf-4ddc-a1d7-96484674db68 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:02.189469 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d178d87-a5cf-4ddc-a1d7-96484674db68 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:02.189798 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d178d87-a5cf-4ddc-a1d7-96484674db68 None admin] PATH_INFO: `/v3/services/image` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:02.190373 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d178d87-a5cf-4ddc-a1d7-96484674db68 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:02.191752 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d178d87-a5cf-4ddc-a1d7-96484674db68 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:02.200424 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-8d178d87-a5cf-4ddc-a1d7-96484674db68 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 20:16:02.200424 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:16:02.200424 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:16:02.200424 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:16:02.200424 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:16:02.200424 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:16:02.200424 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:16:02.200424 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:16:02.200424 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 20:16:02.200424 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:16:02.200424 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:16:02.200424 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:16:02.200424 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:16:02.200424 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:16:02.200424 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:16:02.200424 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:16:02.200424 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:16:02.201726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:16:02.201726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:16:02.201726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:16:02.201726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:16:02.201726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:16:02.201726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:16:02.201726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:16:02.201726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:16:02.201726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:16:02.201726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:16:02.201726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:16:02.201726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:16:02.201726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:16:02.201726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:16:02.201726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:16:02.201726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:16:02.202612 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:16:02.202612 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:16:02.202612 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:16:02.202612 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:16:02.202612 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:16:02.202612 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:16:02.202612 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:16:02.202612 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:16:02.202612 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:16:02.202612 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:16:02.202612 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:16:02.202612 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:16:02.202612 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 20:16:02.202612 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:16:02.205559 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 529/1058] 10.222.0.26 () {56 vars in 1105 bytes} [Fri Sep 9 20:16:02 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:16:02.207347 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1885ee6-83dd-4497-afc3-d8b00d958856 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:02.276294 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:02.307446 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1885ee6-83dd-4497-afc3-d8b00d958856 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:02.339915 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3fb81-d68f-47d2-a599-bf76cd20c24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:02.340519 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3fb81-d68f-47d2-a599-bf76cd20c24e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:02.340794 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3fb81-d68f-47d2-a599-bf76cd20c24e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:02.341064 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3fb81-d68f-47d2-a599-bf76cd20c24e None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:02.341712 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b3fb81-d68f-47d2-a599-bf76cd20c24e None admin] RBAC: Adding query filter params (name=image) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:16:02.342011 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b3fb81-d68f-47d2-a599-bf76cd20c24e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:02.343351 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b3fb81-d68f-47d2-a599-bf76cd20c24e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:02.348822 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 530/1059] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:16:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:02.354649 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d178d87-a5cf-4ddc-a1d7-96484674db68 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:02.360719 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3fb81-d68f-47d2-a599-bf76cd20c24e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:02.466478 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3fb81-d68f-47d2-a599-bf76cd20c24e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:02.469913 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d178d87-a5cf-4ddc-a1d7-96484674db68 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:02.499860 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451ef0a-3160-41a5-8631-2ffc37b0a36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:02.500542 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451ef0a-3160-41a5-8631-2ffc37b0a36e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:02.500830 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451ef0a-3160-41a5-8631-2ffc37b0a36e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:02.501100 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451ef0a-3160-41a5-8631-2ffc37b0a36e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:02.503975 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:02.508000 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90a1c9-8490-4ed4-b364-87c9e0e4371c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:02.509325 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90a1c9-8490-4ed4-b364-87c9e0e4371c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:02.509839 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90a1c9-8490-4ed4-b364-87c9e0e4371c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:02.510288 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90a1c9-8490-4ed4-b364-87c9e0e4371c None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:02.511109 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df90a1c9-8490-4ed4-b364-87c9e0e4371c None admin] RBAC: Adding query filter params (type=image) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:16:02.511585 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df90a1c9-8490-4ed4-b364-87c9e0e4371c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:02.512736 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c451ef0a-3160-41a5-8631-2ffc37b0a36e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:16:02.513420 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df90a1c9-8490-4ed4-b364-87c9e0e4371c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:02.517512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-c451ef0a-3160-41a5-8631-2ffc37b0a36e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 20:16:02.517512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:16:02.517512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:16:02.517512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:16:02.517512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:16:02.517512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:16:02.517512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:16:02.517512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:16:02.517512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:16:02.517512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:16:02.517512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:16:02.517512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:16:02.517512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:16:02.517512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:16:02.517512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:16:02.517512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:16:02.518468 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:16:02.518468 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:16:02.518468 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:16:02.518468 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:16:02.518468 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:16:02.518468 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:16:02.518468 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:16:02.518468 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:16:02.518468 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:16:02.518468 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:16:02.518468 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:16:02.518468 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:16:02.518468 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:16:02.518468 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:16:02.518468 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:16:02.518468 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:16:02.519347 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:16:02.519347 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:16:02.519347 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:16:02.519347 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:16:02.519347 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:16:02.519347 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:16:02.519347 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:16:02.519347 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:16:02.519347 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:16:02.519347 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:16:02.519347 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 20:16:02.519347 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:16:02.519347 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 531/1060] 10.222.0.26 () {56 vars in 1120 bytes} [Fri Sep 9 20:16:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:16:02.522105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 530/1061] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:16:02 2022] GET /identity/v3/services?type=image => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:16:02.524890 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451ef0a-3160-41a5-8631-2ffc37b0a36e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:02.527365 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90a1c9-8490-4ed4-b364-87c9e0e4371c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:02.585200 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:02.630626 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451ef0a-3160-41a5-8631-2ffc37b0a36e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:02.631554 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90a1c9-8490-4ed4-b364-87c9e0e4371c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:02.670085 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cabc2bf-dc03-4471-aa2e-8a7501d80c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:02.670706 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabc2bf-dc03-4471-aa2e-8a7501d80c71 None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:02.670986 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabc2bf-dc03-4471-aa2e-8a7501d80c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:02.671261 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabc2bf-dc03-4471-aa2e-8a7501d80c71 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:02.671938 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cabc2bf-dc03-4471-aa2e-8a7501d80c71 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:02.673353 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cabc2bf-dc03-4471-aa2e-8a7501d80c71 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:02.674866 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf18f45c-1b4d-4cdb-93b3-40592b3be4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:02.675479 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18f45c-1b4d-4cdb-93b3-40592b3be4bc None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:02.675757 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18f45c-1b4d-4cdb-93b3-40592b3be4bc None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:02.676055 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18f45c-1b4d-4cdb-93b3-40592b3be4bc None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:02.676723 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf18f45c-1b4d-4cdb-93b3-40592b3be4bc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:16:02.677024 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf18f45c-1b4d-4cdb-93b3-40592b3be4bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:02.678414 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf18f45c-1b4d-4cdb-93b3-40592b3be4bc None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:02.687707 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 532/1062] 10.222.0.26 () {56 vars in 1115 bytes} [Fri Sep 9 20:16:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:16:02.693112 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf18f45c-1b4d-4cdb-93b3-40592b3be4bc None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:02.696880 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 531/1063] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:16:02 2022] POST /identity/v3/endpoints => generated 327 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:16:02.761954 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:02.813180 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf18f45c-1b4d-4cdb-93b3-40592b3be4bc None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:02.857873 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43598b87-6026-425a-886a-aa2df0ab640c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:02.858517 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43598b87-6026-425a-886a-aa2df0ab640c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:02.858796 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43598b87-6026-425a-886a-aa2df0ab640c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:02.859068 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43598b87-6026-425a-886a-aa2df0ab640c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:02.870892 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43598b87-6026-425a-886a-aa2df0ab640c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:02.872326 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43598b87-6026-425a-886a-aa2df0ab640c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:02.889239 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-43598b87-6026-425a-886a-aa2df0ab640c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:16:02.889239 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:16:02.889239 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:16:02.889239 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:16:02.889239 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:16:02.889239 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:16:02.889239 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:16:02.889239 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:16:02.889239 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:16:02.889239 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:16:02.889239 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:16:02.889239 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:16:02.889239 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:16:02.889239 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:16:02.889239 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:16:02.889239 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:16:02.890274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:16:02.890274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:16:02.890274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:16:02.890274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:16:02.890274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:16:02.890274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:16:02.890274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:16:02.890274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:16:02.890274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:16:02.890274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:16:02.890274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:16:02.890274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:16:02.890274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:16:02.890274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:16:02.890274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:16:02.890274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:16:02.890274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:16:02.891243 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:16:02.891243 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:16:02.891243 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:16:02.891243 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 533/1064] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:16:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:16:02.897037 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cabc2bf-dc03-4471-aa2e-8a7501d80c71 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:03.000279 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cabc2bf-dc03-4471-aa2e-8a7501d80c71 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:03.010837 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:03.038482 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b74bc-a810-44f9-8ec5-f269884f1634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:03.039019 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b74bc-a810-44f9-8ec5-f269884f1634 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:03.039293 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b74bc-a810-44f9-8ec5-f269884f1634 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:03.039561 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b74bc-a810-44f9-8ec5-f269884f1634 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:03.040197 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b74bc-a810-44f9-8ec5-f269884f1634 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:16:03.040494 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b74bc-a810-44f9-8ec5-f269884f1634 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:03.041876 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b74bc-a810-44f9-8ec5-f269884f1634 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:03.055280 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 532/1065] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:03.061592 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43598b87-6026-425a-886a-aa2df0ab640c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:03.166799 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43598b87-6026-425a-886a-aa2df0ab640c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:03.174040 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:03.203004 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac5918-f0d5-4391-88fe-a3e6d7ce8120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:03.203644 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac5918-f0d5-4391-88fe-a3e6d7ce8120 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:03.203920 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac5918-f0d5-4391-88fe-a3e6d7ce8120 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:03.204190 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac5918-f0d5-4391-88fe-a3e6d7ce8120 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:03.211386 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac5918-f0d5-4391-88fe-a3e6d7ce8120 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:03.212839 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac5918-f0d5-4391-88fe-a3e6d7ce8120 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:03.225433 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-0dac5918-f0d5-4391-88fe-a3e6d7ce8120 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:16:03.225433 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:16:03.225433 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:16:03.225433 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:16:03.225433 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:16:03.225433 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:16:03.225433 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:16:03.225433 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:16:03.225433 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:16:03.225433 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:16:03.225433 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:16:03.225433 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:16:03.225433 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:16:03.225433 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:16:03.225433 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:16:03.225433 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:16:03.226452 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:16:03.226452 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:16:03.226452 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:16:03.226452 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:16:03.226452 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:16:03.226452 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:16:03.226452 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:16:03.226452 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:16:03.226452 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:16:03.226452 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:16:03.226452 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:16:03.226452 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:16:03.226452 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:16:03.226452 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:16:03.226452 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:16:03.226452 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:16:03.226452 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:16:03.226452 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:16:03.227427 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:16:03.227427 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:16:03.227427 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:16:03.227427 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:16:03.227427 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:16:03.227427 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:16:03.227427 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:16:03.227427 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:16:03.227427 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:16:03.227427 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:16:03.227427 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 534/1066] 10.222.0.26 () {56 vars in 1093 bytes} [Fri Sep 9 20:16:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:16:03.234499 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b74bc-a810-44f9-8ec5-f269884f1634 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:03.337062 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b74bc-a810-44f9-8ec5-f269884f1634 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:03.373412 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad96e4-ed1f-4ede-bd72-986404c99a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:03.374171 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad96e4-ed1f-4ede-bd72-986404c99a16 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:03.374557 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad96e4-ed1f-4ede-bd72-986404c99a16 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:03.374921 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad96e4-ed1f-4ede-bd72-986404c99a16 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:03.375740 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ad96e4-ed1f-4ede-bd72-986404c99a16 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:16:03.376166 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ad96e4-ed1f-4ede-bd72-986404c99a16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:03.377706 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ad96e4-ed1f-4ede-bd72-986404c99a16 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:03.397412 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 533/1067] 10.222.0.26 () {56 vars in 1133 bytes} [Fri Sep 9 20:16:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:03.401633 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac5918-f0d5-4391-88fe-a3e6d7ce8120 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:03.503110 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac5918-f0d5-4391-88fe-a3e6d7ce8120 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:03.537925 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127c551-131c-4691-aa24-cf2d5a3caa82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:03.538558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127c551-131c-4691-aa24-cf2d5a3caa82 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:03.538834 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127c551-131c-4691-aa24-cf2d5a3caa82 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:03.539112 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127c551-131c-4691-aa24-cf2d5a3caa82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:03.549632 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9127c551-131c-4691-aa24-cf2d5a3caa82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:03.551600 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9127c551-131c-4691-aa24-cf2d5a3caa82 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:03.565559 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-9127c551-131c-4691-aa24-cf2d5a3caa82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:16:03.565559 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:16:03.565559 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:16:03.565559 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:16:03.565559 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:16:03.565559 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:16:03.565559 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:16:03.565559 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:16:03.565559 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:16:03.565559 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:16:03.565559 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:16:03.565559 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:16:03.565559 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:16:03.565559 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:16:03.565559 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:16:03.565559 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:16:03.566579 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:16:03.566579 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:16:03.566579 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:16:03.566579 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:16:03.566579 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:16:03.566579 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:16:03.566579 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:16:03.566579 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:16:03.566579 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:16:03.566579 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:16:03.566579 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:16:03.566579 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:16:03.566579 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:16:03.566579 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:16:03.566579 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:16:03.566579 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:16:03.566579 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:16:03.567517 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:16:03.567517 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:16:03.567517 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:16:03.567517 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 535/1068] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:16:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:16:03.574481 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad96e4-ed1f-4ede-bd72-986404c99a16 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:03.604349 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:03.689474 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad96e4-ed1f-4ede-bd72-986404c99a16 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:03.724230 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc98523-9b0a-44f2-958d-3d6cc512237a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:03.724833 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc98523-9b0a-44f2-958d-3d6cc512237a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:03.725112 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc98523-9b0a-44f2-958d-3d6cc512237a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:03.725425 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc98523-9b0a-44f2-958d-3d6cc512237a None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:03.726063 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc98523-9b0a-44f2-958d-3d6cc512237a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:16:03.726363 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc98523-9b0a-44f2-958d-3d6cc512237a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:03.727717 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc98523-9b0a-44f2-958d-3d6cc512237a None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:03.739773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 534/1069] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:03.746794 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127c551-131c-4691-aa24-cf2d5a3caa82 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:03.767369 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:03.848236 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127c551-131c-4691-aa24-cf2d5a3caa82 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:03.882040 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166f8bb-8677-402f-aee5-7e3924f475ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:03.882601 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166f8bb-8677-402f-aee5-7e3924f475ce None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:03.882881 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166f8bb-8677-402f-aee5-7e3924f475ce None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:03.883150 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166f8bb-8677-402f-aee5-7e3924f475ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:03.891449 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e166f8bb-8677-402f-aee5-7e3924f475ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:03.891449 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e166f8bb-8677-402f-aee5-7e3924f475ce None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:03.907990 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-e166f8bb-8677-402f-aee5-7e3924f475ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:16:03.907990 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:16:03.907990 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:16:03.907990 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:16:03.907990 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:16:03.907990 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:16:03.907990 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:16:03.907990 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:16:03.907990 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:16:03.907990 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:16:03.907990 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:16:03.907990 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:16:03.907990 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:16:03.907990 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:16:03.907990 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:16:03.909023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:16:03.909023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:16:03.909023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:16:03.909023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:16:03.909023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:16:03.909023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:16:03.909023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:16:03.909023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:16:03.909023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:16:03.909023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:16:03.909023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:16:03.909023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:16:03.909023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:16:03.909023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:16:03.909023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:16:03.909023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:16:03.909023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:16:03.910571 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:16:03.910571 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:16:03.910571 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:16:03.910571 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:16:03.910571 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:16:03.910571 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:16:03.910571 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:16:03.910571 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:16:03.913451 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 536/1070] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:16:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:16:03.914955 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc98523-9b0a-44f2-958d-3d6cc512237a None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:04.008692 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:04.025233 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc98523-9b0a-44f2-958d-3d6cc512237a None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:04.061340 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8817ee14-148e-473c-abc0-10ea0f89cab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:04.062032 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817ee14-148e-473c-abc0-10ea0f89cab4 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:04.062380 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817ee14-148e-473c-abc0-10ea0f89cab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:04.062717 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817ee14-148e-473c-abc0-10ea0f89cab4 None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:04.063522 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8817ee14-148e-473c-abc0-10ea0f89cab4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:16:04.063893 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8817ee14-148e-473c-abc0-10ea0f89cab4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:04.065406 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8817ee14-148e-473c-abc0-10ea0f89cab4 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:04.079405 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 535/1071] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:16:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:16:04.084520 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166f8bb-8677-402f-aee5-7e3924f475ce None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:04.122452 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105bd82-1bf6-4ed3-aa1f-3c3eceea7ba1 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:04.122731 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105bd82-1bf6-4ed3-aa1f-3c3eceea7ba1 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:04.123002 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105bd82-1bf6-4ed3-aa1f-3c3eceea7ba1 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:04.124465 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 536/1072] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:16:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:16:04.130521 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9331cca2-abe3-4225-b427-ef43e2f5e4e1 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:04.131121 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9331cca2-abe3-4225-b427-ef43e2f5e4e1 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:04.131395 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9331cca2-abe3-4225-b427-ef43e2f5e4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:04.149000 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-9331cca2-abe3-4225-b427-ef43e2f5e4e1 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:16:04.150153 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-9331cca2-abe3-4225-b427-ef43e2f5e4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:16:04.170978 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:04.196289 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166f8bb-8677-402f-aee5-7e3924f475ce None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:04.223118 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 537/1073] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:16:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:16:04.232236 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d2751-2453-47bc-9f3a-c8dc833a9070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:04.232889 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d2751-2453-47bc-9f3a-c8dc833a9070 None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:04.233176 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d2751-2453-47bc-9f3a-c8dc833a9070 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:04.233676 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d2751-2453-47bc-9f3a-c8dc833a9070 None admin] PATH_INFO: `/v3/projects/933820e2adbc473bbd57a4d59be75d72/users/87ece6128f9245f7b49f3e7a9faaed22/roles/6f8d308c22cc406ba07771a051b4bde2` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:04.235820 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02d2751-2453-47bc-9f3a-c8dc833a9070 None admin] RBAC: Authorizing `identity:create_grant(project_id=933820e2adbc473bbd57a4d59be75d72, user_id=87ece6128f9245f7b49f3e7a9faaed22, role_id=6f8d308c22cc406ba07771a051b4bde2)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:04.237231 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02d2751-2453-47bc-9f3a-c8dc833a9070 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:04.252038 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 537/1074] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Sep 9 20:16:04 2022] PUT /identity/v3/projects/933820e2adbc473bbd57a4d59be75d72/users/87ece6128f9245f7b49f3e7a9faaed22/roles/6f8d308c22cc406ba07771a051b4bde2 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:16:04.491974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9331cca2-abe3-4225-b427-ef43e2f5e4e1 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:04.572366 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:04.594998 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9331cca2-abe3-4225-b427-ef43e2f5e4e1 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:04.631166 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2384cfb2-48a3-4526-86d2-9969e6962ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:04.631808 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384cfb2-48a3-4526-86d2-9969e6962ff4 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:04.632090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384cfb2-48a3-4526-86d2-9969e6962ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:04.632361 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384cfb2-48a3-4526-86d2-9969e6962ff4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:04.642556 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2384cfb2-48a3-4526-86d2-9969e6962ff4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:16:04.647854 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-2384cfb2-48a3-4526-86d2-9969e6962ff4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:16:04.647854 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:16:04.647854 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:16:04.647854 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:16:04.647854 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:16:04.647854 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:16:04.647854 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:16:04.647854 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:16:04.647854 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:16:04.647854 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:16:04.647854 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:16:04.647854 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:16:04.647854 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:16:04.647854 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:16:04.647854 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:16:04.647854 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:16:04.648806 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:16:04.648806 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:16:04.648806 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:16:04.648806 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:16:04.648806 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:16:04.648806 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:16:04.648806 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:16:04.648806 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:16:04.648806 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:16:04.648806 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:16:04.648806 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:16:04.648806 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:16:04.648806 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:16:04.648806 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:16:04.648806 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:16:04.648806 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:16:04.649748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:16:04.649748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:16:04.649748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:16:04.649748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:16:04.649748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:16:04.649748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:16:04.649748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:16:04.649748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:16:04.649748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:16:04.649748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:16:04.649748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:16:04.649748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:16:04.651023 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 538/1075] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:16:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:16:04.655484 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d2751-2453-47bc-9f3a-c8dc833a9070 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:04.768688 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d2751-2453-47bc-9f3a-c8dc833a9070 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:04.807400 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afbc0dd-c940-4eff-b20d-381d9f1b33a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:04.808022 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afbc0dd-c940-4eff-b20d-381d9f1b33a0 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:04.808304 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afbc0dd-c940-4eff-b20d-381d9f1b33a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:04.808578 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afbc0dd-c940-4eff-b20d-381d9f1b33a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:04.809270 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afbc0dd-c940-4eff-b20d-381d9f1b33a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:16:04.809577 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afbc0dd-c940-4eff-b20d-381d9f1b33a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:04.810977 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afbc0dd-c940-4eff-b20d-381d9f1b33a0 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:04.816659 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:04.826883 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 538/1076] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:16:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:04.830316 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2384cfb2-48a3-4526-86d2-9969e6962ff4 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:04.899712 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725d622-60ca-4d6c-bfd6-fc1d2cbc6d7d None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:04.899998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725d622-60ca-4d6c-bfd6-fc1d2cbc6d7d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:04.900267 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725d622-60ca-4d6c-bfd6-fc1d2cbc6d7d None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:04.901777 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 539/1077] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:16:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:16:04.906563 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f29db8-ae7f-4e86-a378-51fcc22cd4a9 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:04.906843 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f29db8-ae7f-4e86-a378-51fcc22cd4a9 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:04.907112 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f29db8-ae7f-4e86-a378-51fcc22cd4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:04.930032 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-b3f29db8-ae7f-4e86-a378-51fcc22cd4a9 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:16:04.931410 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b3f29db8-ae7f-4e86-a378-51fcc22cd4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:16:04.933547 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2384cfb2-48a3-4526-86d2-9969e6962ff4 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:04.974215 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f5a45-1467-4530-adae-bbc5be13fdbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:04.974851 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f5a45-1467-4530-adae-bbc5be13fdbc None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:04.975125 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f5a45-1467-4530-adae-bbc5be13fdbc None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:04.975444 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f5a45-1467-4530-adae-bbc5be13fdbc None admin] PATH_INFO: `/v3/users/7adec72566154be7b81d03fa02949afb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:04.977009 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637f5a45-1467-4530-adae-bbc5be13fdbc None admin] RBAC: Authorizing `identity:get_user(user_id=7adec72566154be7b81d03fa02949afb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:04.978831 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637f5a45-1467-4530-adae-bbc5be13fdbc None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:04.981048 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 539/1078] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:16:04 2022] GET /identity/v3/users/7adec72566154be7b81d03fa02949afb => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:04.986710 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f5a45-1467-4530-adae-bbc5be13fdbc None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:04.995614 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:05.014034 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 540/1079] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:16:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:16:05.094833 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f5a45-1467-4530-adae-bbc5be13fdbc None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:05.127114 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b413996-7ba9-4eb3-8dd9-b31f2b21f6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:05.127748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b413996-7ba9-4eb3-8dd9-b31f2b21f6a6 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:05.128026 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b413996-7ba9-4eb3-8dd9-b31f2b21f6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:05.128308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b413996-7ba9-4eb3-8dd9-b31f2b21f6a6 None admin] PATH_INFO: `/v3/projects/81d44cfe705d437987b23852c51e07cc` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:05.130261 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b413996-7ba9-4eb3-8dd9-b31f2b21f6a6 None admin] RBAC: Authorizing `identity:get_project(project_id=81d44cfe705d437987b23852c51e07cc)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:05.132543 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b413996-7ba9-4eb3-8dd9-b31f2b21f6a6 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:05.135311 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 540/1080] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:16:04 2022] GET /identity/v3/projects/81d44cfe705d437987b23852c51e07cc => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:16:05.141629 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f29db8-ae7f-4e86-a378-51fcc22cd4a9 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:05.243145 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f29db8-ae7f-4e86-a378-51fcc22cd4a9 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:05.270999 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:05.278049 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6781e4a7-f6fa-4c0b-bc9d-a913e4292938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:05.278707 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781e4a7-f6fa-4c0b-bc9d-a913e4292938 None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:05.278986 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781e4a7-f6fa-4c0b-bc9d-a913e4292938 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:05.279259 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781e4a7-f6fa-4c0b-bc9d-a913e4292938 None admin] PATH_INFO: `/v3/projects/81d44cfe705d437987b23852c51e07cc/users/7adec72566154be7b81d03fa02949afb/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:05.281134 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6781e4a7-f6fa-4c0b-bc9d-a913e4292938 None admin] RBAC: Authorizing `identity:create_grant(project_id=81d44cfe705d437987b23852c51e07cc, user_id=7adec72566154be7b81d03fa02949afb, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:05.282574 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6781e4a7-f6fa-4c0b-bc9d-a913e4292938 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:05.285839 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b413996-7ba9-4eb3-8dd9-b31f2b21f6a6 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:05.297693 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 541/1081] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Sep 9 20:16:05 2022] PUT /identity/v3/projects/81d44cfe705d437987b23852c51e07cc/users/7adec72566154be7b81d03fa02949afb/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:16:05.373990 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:05.403129 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b413996-7ba9-4eb3-8dd9-b31f2b21f6a6 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:05.439549 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b3eecd-7fbc-446e-b499-9a4467ab1c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:05.440168 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3eecd-7fbc-446e-b499-9a4467ab1c56 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:05.440445 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3eecd-7fbc-446e-b499-9a4467ab1c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:05.440720 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3eecd-7fbc-446e-b499-9a4467ab1c56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:05.516075 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b3eecd-7fbc-446e-b499-9a4467ab1c56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:05.517609 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b3eecd-7fbc-446e-b499-9a4467ab1c56 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:05.591682 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:05.618733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 541/1082] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:16:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:16:05.624410 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6781e4a7-f6fa-4c0b-bc9d-a913e4292938 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:05.733213 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6781e4a7-f6fa-4c0b-bc9d-a913e4292938 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:05.770615 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ebe39-bfd0-470e-9ea2-388d2acd7682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:05.771278 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ebe39-bfd0-470e-9ea2-388d2acd7682 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:05.771501 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ebe39-bfd0-470e-9ea2-388d2acd7682 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:05.771773 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ebe39-bfd0-470e-9ea2-388d2acd7682 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:05.782917 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8ebe39-bfd0-470e-9ea2-388d2acd7682 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:05.784352 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8ebe39-bfd0-470e-9ea2-388d2acd7682 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:05.798308 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-9e8ebe39-bfd0-470e-9ea2-388d2acd7682 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:16:05.798308 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:16:05.798308 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:16:05.798308 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:16:05.798308 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:16:05.798308 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:16:05.798308 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:16:05.798308 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:16:05.798308 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:16:05.798308 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:16:05.798308 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:16:05.798308 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:16:05.798308 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:16:05.798308 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:16:05.798308 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:16:05.798308 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:16:05.799323 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:16:05.799323 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:16:05.799323 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:16:05.799323 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:16:05.799323 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:16:05.799323 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:16:05.799323 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:16:05.799323 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:16:05.799323 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:16:05.799323 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:16:05.799323 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:16:05.799323 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:16:05.799323 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:16:05.799323 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:16:05.799323 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:16:05.799323 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:16:05.799323 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:16:05.800261 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:16:05.800261 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:16:05.800261 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:16:05.801162 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 542/1083] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:16:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:16:05.805400 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b3eecd-7fbc-446e-b499-9a4467ab1c56 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:05.904107 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b3eecd-7fbc-446e-b499-9a4467ab1c56 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:05.937070 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe25a30-ad57-489f-bbb5-abf57cc873d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:05.937701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe25a30-ad57-489f-bbb5-abf57cc873d6 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:05.937978 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe25a30-ad57-489f-bbb5-abf57cc873d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:05.938247 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe25a30-ad57-489f-bbb5-abf57cc873d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:05.938880 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe25a30-ad57-489f-bbb5-abf57cc873d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:16:05.939179 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe25a30-ad57-489f-bbb5-abf57cc873d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:05.940529 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe25a30-ad57-489f-bbb5-abf57cc873d6 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:05.961291 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 542/1084] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:16:06.443353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44332b21-29c7-4b85-871e-071dbc93ed60 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:06.443751 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44332b21-29c7-4b85-871e-071dbc93ed60 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:06.444037 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44332b21-29c7-4b85-871e-071dbc93ed60 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:06.445387 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 543/1085] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:16:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:16:06.450712 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b722e4-371e-4f57-bb4a-7b8b1b479cef None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:06.450996 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b722e4-371e-4f57-bb4a-7b8b1b479cef None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:06.451265 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b722e4-371e-4f57-bb4a-7b8b1b479cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:06.469551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-69b722e4-371e-4f57-bb4a-7b8b1b479cef None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:16:06.470718 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-69b722e4-371e-4f57-bb4a-7b8b1b479cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:16:06.496123 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:06.536541 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 543/1086] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:16:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:16:06.805712 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44332b21-29c7-4b85-871e-071dbc93ed60 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:06.848900 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:06.906622 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44332b21-29c7-4b85-871e-071dbc93ed60 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:06.939939 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3866b5-841c-4e03-a1bc-8c46db4438bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:06.940555 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3866b5-841c-4e03-a1bc-8c46db4438bd None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:06.940832 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3866b5-841c-4e03-a1bc-8c46db4438bd None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:06.941126 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3866b5-841c-4e03-a1bc-8c46db4438bd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:06.950937 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3866b5-841c-4e03-a1bc-8c46db4438bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:16:06.955614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-3e3866b5-841c-4e03-a1bc-8c46db4438bd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 20:16:06.955614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:16:06.955614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:16:06.955614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:16:06.955614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:16:06.955614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:16:06.955614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:16:06.955614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:16:06.955614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:16:06.955614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:16:06.955614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:16:06.955614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:16:06.955614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:16:06.955614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:16:06.955614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:16:06.955614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:16:06.956618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:16:06.956618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:16:06.956618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:16:06.956618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:16:06.956618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:16:06.956618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:16:06.956618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:16:06.956618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:16:06.956618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:16:06.956618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:16:06.956618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:16:06.956618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:16:06.956618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:16:06.956618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:16:06.956618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:16:06.956618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:16:06.958062 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:16:06.958062 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:16:06.958062 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:16:06.958062 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:16:06.958062 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:16:06.958062 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:16:06.958062 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:16:06.958062 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:16:06.958062 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:16:06.958062 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:16:06.958062 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 20:16:06.958062 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:16:06.958062 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 544/1087] 10.222.0.26 () {56 vars in 1120 bytes} [Fri Sep 9 20:16:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:16:06.963137 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b722e4-371e-4f57-bb4a-7b8b1b479cef None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:07.059205 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b722e4-371e-4f57-bb4a-7b8b1b479cef None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:07.091857 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7c263-604d-4cc3-ade6-0a00021390a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:07.092472 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7c263-604d-4cc3-ade6-0a00021390a0 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:07.092737 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7c263-604d-4cc3-ade6-0a00021390a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:07.093009 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7c263-604d-4cc3-ade6-0a00021390a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:07.093704 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce7c263-604d-4cc3-ade6-0a00021390a0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:16:07.094006 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce7c263-604d-4cc3-ade6-0a00021390a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:07.095372 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce7c263-604d-4cc3-ade6-0a00021390a0 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:07.103688 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 544/1088] 10.222.0.26 () {56 vars in 1115 bytes} [Fri Sep 9 20:16:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:07.109612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3866b5-841c-4e03-a1bc-8c46db4438bd None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:07.211539 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3866b5-841c-4e03-a1bc-8c46db4438bd None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:07.213077 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:07.245984 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c36cdb-9391-4f4d-88fb-0d05cba9c1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:07.246609 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c36cdb-9391-4f4d-88fb-0d05cba9c1bc None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:07.246885 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c36cdb-9391-4f4d-88fb-0d05cba9c1bc None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:07.247153 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c36cdb-9391-4f4d-88fb-0d05cba9c1bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:07.257614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c36cdb-9391-4f4d-88fb-0d05cba9c1bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:07.259025 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c36cdb-9391-4f4d-88fb-0d05cba9c1bc None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:07.273632 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-73c36cdb-9391-4f4d-88fb-0d05cba9c1bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:16:07.273632 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:16:07.273632 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:16:07.273632 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:16:07.273632 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:16:07.273632 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:16:07.273632 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:16:07.273632 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:16:07.273632 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:16:07.273632 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:16:07.273632 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:16:07.273632 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:16:07.273632 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:16:07.273632 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:16:07.273632 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:16:07.273632 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:16:07.274634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:16:07.274634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:16:07.274634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:16:07.274634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:16:07.274634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:16:07.274634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:16:07.274634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:16:07.274634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:16:07.274634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:16:07.274634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:16:07.274634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:16:07.274634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:16:07.274634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:16:07.274634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:16:07.274634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:16:07.274634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:16:07.274634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:16:07.275565 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:16:07.275565 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:16:07.275565 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:16:07.275565 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 545/1089] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:16:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:16:07.280390 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7c263-604d-4cc3-ade6-0a00021390a0 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:07.307322 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:07.382406 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7c263-604d-4cc3-ade6-0a00021390a0 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:07.416363 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edce28b6-ba26-4694-ae17-3a0f47fb7f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:07.416967 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce28b6-ba26-4694-ae17-3a0f47fb7f50 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:07.417276 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce28b6-ba26-4694-ae17-3a0f47fb7f50 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:07.417542 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce28b6-ba26-4694-ae17-3a0f47fb7f50 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:07.418180 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edce28b6-ba26-4694-ae17-3a0f47fb7f50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:16:07.418481 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edce28b6-ba26-4694-ae17-3a0f47fb7f50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:07.419837 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edce28b6-ba26-4694-ae17-3a0f47fb7f50 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:07.431317 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 545/1090] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:16:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:07.437023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c36cdb-9391-4f4d-88fb-0d05cba9c1bc None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:07.478061 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d297e-f542-4c5e-8f49-8ec47d3fa18d None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:07.478461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d297e-f542-4c5e-8f49-8ec47d3fa18d None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:07.478736 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d297e-f542-4c5e-8f49-8ec47d3fa18d None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:07.480061 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 546/1091] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:16:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:16:07.485018 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d65a78-7e6a-45d5-ba19-cb9a9b50b151 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:07.485343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d65a78-7e6a-45d5-ba19-cb9a9b50b151 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:07.485619 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d65a78-7e6a-45d5-ba19-cb9a9b50b151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:07.503151 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-16d65a78-7e6a-45d5-ba19-cb9a9b50b151 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:16:07.505138 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-16d65a78-7e6a-45d5-ba19-cb9a9b50b151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:16:07.534946 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c36cdb-9391-4f4d-88fb-0d05cba9c1bc None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:07.567790 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd6a31-3478-4b9f-a9a3-16a309493161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:07.568418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd6a31-3478-4b9f-a9a3-16a309493161 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:07.568700 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd6a31-3478-4b9f-a9a3-16a309493161 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:07.568970 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd6a31-3478-4b9f-a9a3-16a309493161 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:07.570711 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 547/1092] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:16:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:16:07.576706 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd6a31-3478-4b9f-a9a3-16a309493161 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:07.578158 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd6a31-3478-4b9f-a9a3-16a309493161 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:07.589808 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-6bbd6a31-3478-4b9f-a9a3-16a309493161 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:16:07.589808 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:16:07.589808 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:16:07.589808 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:16:07.589808 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:16:07.589808 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:16:07.589808 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:16:07.589808 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:16:07.589808 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:16:07.589808 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:16:07.589808 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:16:07.589808 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:16:07.589808 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:16:07.589808 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:16:07.589808 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:16:07.589808 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:16:07.590783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:16:07.590783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:16:07.590783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:16:07.590783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:16:07.590783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:16:07.590783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:16:07.590783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:16:07.590783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:16:07.590783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:16:07.590783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:16:07.590783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:16:07.590783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:16:07.590783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:16:07.590783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:16:07.590783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:16:07.590783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:16:07.590783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:16:07.590783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:16:07.591763 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:16:07.591763 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:16:07.591763 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:16:07.591763 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:16:07.591763 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:16:07.591763 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:16:07.591763 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:16:07.591763 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:16:07.591763 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:16:07.591763 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:16:07.591763 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 546/1093] 10.222.0.26 () {56 vars in 1093 bytes} [Fri Sep 9 20:16:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:16:07.596694 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d65a78-7e6a-45d5-ba19-cb9a9b50b151 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:07.613943 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:07.697048 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d65a78-7e6a-45d5-ba19-cb9a9b50b151 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:07.729449 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7605843c-c6ee-4f71-a6d5-41ec6f6ca11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:07.730055 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605843c-c6ee-4f71-a6d5-41ec6f6ca11c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:07.730331 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605843c-c6ee-4f71-a6d5-41ec6f6ca11c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:07.730601 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605843c-c6ee-4f71-a6d5-41ec6f6ca11c None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:07.731318 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7605843c-c6ee-4f71-a6d5-41ec6f6ca11c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:16:07.731619 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7605843c-c6ee-4f71-a6d5-41ec6f6ca11c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:07.733045 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7605843c-c6ee-4f71-a6d5-41ec6f6ca11c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:07.750023 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 548/1094] 10.222.0.26 () {56 vars in 1133 bytes} [Fri Sep 9 20:16:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:07.756748 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd6a31-3478-4b9f-a9a3-16a309493161 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:07.773821 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:07.845209 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7605843c-c6ee-4f71-a6d5-41ec6f6ca11c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:07.857033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd6a31-3478-4b9f-a9a3-16a309493161 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:07.891482 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1efad5-87af-4d1b-9f5c-c08d63057b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:07.892089 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1efad5-87af-4d1b-9f5c-c08d63057b73 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:07.892374 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1efad5-87af-4d1b-9f5c-c08d63057b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:07.892651 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1efad5-87af-4d1b-9f5c-c08d63057b73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:07.919605 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:07.945370 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7605843c-c6ee-4f71-a6d5-41ec6f6ca11c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:07.961030 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1efad5-87af-4d1b-9f5c-c08d63057b73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:07.962505 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1efad5-87af-4d1b-9f5c-c08d63057b73 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:07.976324 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:07.978152 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204bd58c-6cb1-4ead-b670-69177887b4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:07.978831 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204bd58c-6cb1-4ead-b670-69177887b4c5 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:07.979121 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204bd58c-6cb1-4ead-b670-69177887b4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:07.979347 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204bd58c-6cb1-4ead-b670-69177887b4c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:07.990041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204bd58c-6cb1-4ead-b670-69177887b4c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:16:07.994741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-204bd58c-6cb1-4ead-b670-69177887b4c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:16:07.994741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:16:07.994741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:16:07.994741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:16:07.994741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:16:07.994741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:16:07.994741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:16:07.994741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:16:07.994741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:16:07.994741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:16:07.994741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:16:07.994741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:16:07.994741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:16:07.994741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:16:07.994741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:16:07.994741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:16:07.995721 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:16:07.995721 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:16:07.995721 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:16:07.995721 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:16:07.995721 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:16:07.995721 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:16:07.995721 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:16:07.995721 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:16:07.995721 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:16:07.995721 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:16:07.995721 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:16:07.995721 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:16:07.995721 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:16:07.995721 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:16:07.995721 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:16:07.995721 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:16:07.996630 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:16:07.996630 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:16:07.996630 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:16:07.996630 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:16:07.996630 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:16:07.996630 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:16:07.996630 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:16:07.996630 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:16:07.996630 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:16:07.996630 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:16:07.996630 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:16:07.996630 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:16:07.996630 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 549/1095] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:16:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:16:08.001739 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204bd58c-6cb1-4ead-b670-69177887b4c5 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:08.060194 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 547/1096] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:16:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:16:08.065671 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1efad5-87af-4d1b-9f5c-c08d63057b73 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:08.100458 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204bd58c-6cb1-4ead-b670-69177887b4c5 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:08.133579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7b42e-6bd3-4215-87e3-d7174ea0b60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:08.134189 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7b42e-6bd3-4215-87e3-d7174ea0b60d None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:08.134469 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7b42e-6bd3-4215-87e3-d7174ea0b60d None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:08.134743 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7b42e-6bd3-4215-87e3-d7174ea0b60d None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:08.135403 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b7b42e-6bd3-4215-87e3-d7174ea0b60d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:16:08.135704 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b7b42e-6bd3-4215-87e3-d7174ea0b60d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:08.137073 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b7b42e-6bd3-4215-87e3-d7174ea0b60d None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:08.138434 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:08.145778 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 550/1097] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:16:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:16:08.151533 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7b42e-6bd3-4215-87e3-d7174ea0b60d None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:08.163033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1efad5-87af-4d1b-9f5c-c08d63057b73 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:08.177688 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:08.198659 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dfd197-0def-4bc2-bbe7-4bdd2f0caf5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:08.199231 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dfd197-0def-4bc2-bbe7-4bdd2f0caf5f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:08.199516 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dfd197-0def-4bc2-bbe7-4bdd2f0caf5f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:08.199793 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dfd197-0def-4bc2-bbe7-4bdd2f0caf5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:08.209769 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dfd197-0def-4bc2-bbe7-4bdd2f0caf5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:08.211194 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dfd197-0def-4bc2-bbe7-4bdd2f0caf5f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:08.225895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-81dfd197-0def-4bc2-bbe7-4bdd2f0caf5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:16:08.225895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:16:08.225895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:16:08.225895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:16:08.225895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:16:08.225895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:16:08.225895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:16:08.225895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:16:08.225895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:16:08.225895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:16:08.225895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:16:08.225895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:16:08.225895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:16:08.225895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:16:08.225895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:16:08.225895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:16:08.226850 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:16:08.226850 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:16:08.226850 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:16:08.226850 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:16:08.226850 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:16:08.226850 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:16:08.226850 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:16:08.226850 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:16:08.226850 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:16:08.226850 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:16:08.226850 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:16:08.226850 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:16:08.226850 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:16:08.226850 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:16:08.226850 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:16:08.226850 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:16:08.226850 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:16:08.227816 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:16:08.227816 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:16:08.227816 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:16:08.227816 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 548/1098] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:16:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:16:08.232625 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dfd197-0def-4bc2-bbe7-4bdd2f0caf5f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:08.251397 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7b42e-6bd3-4215-87e3-d7174ea0b60d None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:08.283765 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8bd38d-413e-4bdb-878b-1b89aabe4c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:08.284383 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8bd38d-413e-4bdb-878b-1b89aabe4c2f None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:08.284661 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8bd38d-413e-4bdb-878b-1b89aabe4c2f None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:08.285489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8bd38d-413e-4bdb-878b-1b89aabe4c2f None admin] PATH_INFO: `/v3/users/7adec72566154be7b81d03fa02949afb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:08.286683 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8bd38d-413e-4bdb-878b-1b89aabe4c2f None admin] RBAC: Authorizing `identity:get_user(user_id=7adec72566154be7b81d03fa02949afb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:08.288322 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8bd38d-413e-4bdb-878b-1b89aabe4c2f None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:08.289952 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 551/1099] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:16:08 2022] GET /identity/v3/users/7adec72566154be7b81d03fa02949afb => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:08.296206 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8bd38d-413e-4bdb-878b-1b89aabe4c2f None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:08.331235 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dfd197-0def-4bc2-bbe7-4bdd2f0caf5f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:08.356394 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:08.365564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8348ceb-256a-48cc-867b-d6d7dd29e869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:08.366178 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8348ceb-256a-48cc-867b-d6d7dd29e869 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:08.366456 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8348ceb-256a-48cc-867b-d6d7dd29e869 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:08.366729 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8348ceb-256a-48cc-867b-d6d7dd29e869 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:08.367366 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8348ceb-256a-48cc-867b-d6d7dd29e869 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:16:08.367667 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8348ceb-256a-48cc-867b-d6d7dd29e869 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:08.369016 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8348ceb-256a-48cc-867b-d6d7dd29e869 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:08.380830 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 549/1100] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:16:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:16:08.386428 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8348ceb-256a-48cc-867b-d6d7dd29e869 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:08.396273 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8bd38d-413e-4bdb-878b-1b89aabe4c2f None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:08.412975 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:08.430781 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca71c1-953f-43bb-8204-9ee6f96e7150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:08.431381 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca71c1-953f-43bb-8204-9ee6f96e7150 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:08.431664 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca71c1-953f-43bb-8204-9ee6f96e7150 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:08.431937 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca71c1-953f-43bb-8204-9ee6f96e7150 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:08.484308 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8348ceb-256a-48cc-867b-d6d7dd29e869 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:08.498740 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ca71c1-953f-43bb-8204-9ee6f96e7150 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:08.500179 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ca71c1-953f-43bb-8204-9ee6f96e7150 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:08.517511 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195b03e-98b1-405b-a5a5-a1e6268dd4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:08.518134 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195b03e-98b1-405b-a5a5-a1e6268dd4c0 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:08.518413 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195b03e-98b1-405b-a5a5-a1e6268dd4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:08.518688 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195b03e-98b1-405b-a5a5-a1e6268dd4c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:08.525691 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3195b03e-98b1-405b-a5a5-a1e6268dd4c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:08.527088 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3195b03e-98b1-405b-a5a5-a1e6268dd4c0 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:08.538557 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-3195b03e-98b1-405b-a5a5-a1e6268dd4c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:16:08.538557 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:16:08.538557 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:16:08.538557 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:16:08.538557 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:16:08.538557 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:16:08.538557 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:16:08.538557 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:16:08.538557 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:16:08.538557 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:16:08.538557 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:16:08.538557 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:16:08.538557 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:16:08.538557 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:16:08.538557 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:16:08.539713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:16:08.539713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:16:08.539713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:16:08.539713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:16:08.539713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:16:08.539713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:16:08.539713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:16:08.539713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:16:08.539713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:16:08.539713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:16:08.539713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:16:08.539713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:16:08.539713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:16:08.539713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:16:08.539713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:16:08.539713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:16:08.539713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:16:08.540656 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:16:08.540656 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:16:08.540656 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:16:08.540656 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:16:08.540656 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:16:08.540656 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:16:08.540656 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:16:08.540656 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:16:08.540656 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 550/1101] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:16:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:16:08.545546 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195b03e-98b1-405b-a5a5-a1e6268dd4c0 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:08.561223 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:08.600308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 552/1102] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:16:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 20:16:08.606219 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca71c1-953f-43bb-8204-9ee6f96e7150 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:08.628198 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:08.643633 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195b03e-98b1-405b-a5a5-a1e6268dd4c0 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:08.681538 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf17bb-f634-40c0-8ce0-7d87217624cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:08.682145 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf17bb-f634-40c0-8ce0-7d87217624cd None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:08.682423 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf17bb-f634-40c0-8ce0-7d87217624cd None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:08.682694 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf17bb-f634-40c0-8ce0-7d87217624cd None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:08.683385 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf17bb-f634-40c0-8ce0-7d87217624cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:16:08.683687 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf17bb-f634-40c0-8ce0-7d87217624cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:08.685136 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf17bb-f634-40c0-8ce0-7d87217624cd None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:08.696354 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 551/1103] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:16:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:16:08.702433 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf17bb-f634-40c0-8ce0-7d87217624cd None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:08.710020 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca71c1-953f-43bb-8204-9ee6f96e7150 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:08.743461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd13d1-91d0-4610-a2f4-852950faa276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:08.744023 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd13d1-91d0-4610-a2f4-852950faa276 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:08.744305 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd13d1-91d0-4610-a2f4-852950faa276 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:08.744581 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd13d1-91d0-4610-a2f4-852950faa276 None admin] PATH_INFO: `/v3/projects/81d44cfe705d437987b23852c51e07cc` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:08.745712 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecd13d1-91d0-4610-a2f4-852950faa276 None admin] RBAC: Authorizing `identity:get_project(project_id=81d44cfe705d437987b23852c51e07cc)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:08.747131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecd13d1-91d0-4610-a2f4-852950faa276 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:08.748929 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 553/1104] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:16:08 2022] GET /identity/v3/projects/81d44cfe705d437987b23852c51e07cc => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:16:08.755571 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd13d1-91d0-4610-a2f4-852950faa276 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:08.785367 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:08.803796 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf17bb-f634-40c0-8ce0-7d87217624cd None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:08.837085 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a298003-6c31-4dc3-a9ed-2396085c1bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:08.837729 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a298003-6c31-4dc3-a9ed-2396085c1bf9 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:08.838011 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a298003-6c31-4dc3-a9ed-2396085c1bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:08.838279 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a298003-6c31-4dc3-a9ed-2396085c1bf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:08.838992 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a298003-6c31-4dc3-a9ed-2396085c1bf9 None admin] RBAC: Adding query filter params (role.id=6f8d308c22cc406ba07771a051b4bde2, scope.project.id=933820e2adbc473bbd57a4d59be75d72, user.id=87ece6128f9245f7b49f3e7a9faaed22) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:16:08.839297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a298003-6c31-4dc3-a9ed-2396085c1bf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:08.840701 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a298003-6c31-4dc3-a9ed-2396085c1bf9 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:08.846571 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 552/1105] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:16:08 2022] GET /identity/v3/role_assignments?user.id=87ece6128f9245f7b49f3e7a9faaed22&scope.project.id=933820e2adbc473bbd57a4d59be75d72&role.id=6f8d308c22cc406ba07771a051b4bde2 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:16:08.853187 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd13d1-91d0-4610-a2f4-852950faa276 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:08.890170 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ef5b11-166f-4213-87ad-1ba5a84d7d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:08.890784 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef5b11-166f-4213-87ad-1ba5a84d7d7f None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:08.891058 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef5b11-166f-4213-87ad-1ba5a84d7d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:08.891329 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef5b11-166f-4213-87ad-1ba5a84d7d7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:08.892058 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ef5b11-166f-4213-87ad-1ba5a84d7d7f None admin] RBAC: Adding query filter params (role.id=9121ba25a6d84756990f0c3a19e9fecb, scope.project.id=81d44cfe705d437987b23852c51e07cc, user.id=7adec72566154be7b81d03fa02949afb) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:16:08.892365 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ef5b11-166f-4213-87ad-1ba5a84d7d7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:08.893847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ef5b11-166f-4213-87ad-1ba5a84d7d7f None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:08.901154 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 554/1106] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:16:08 2022] GET /identity/v3/role_assignments?user.id=7adec72566154be7b81d03fa02949afb&scope.project.id=81d44cfe705d437987b23852c51e07cc&role.id=9121ba25a6d84756990f0c3a19e9fecb => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:16:11.093809 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141712ee-7c14-4e4f-a3c9-54478f877976 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:11.094343 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141712ee-7c14-4e4f-a3c9-54478f877976 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:11.094343 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141712ee-7c14-4e4f-a3c9-54478f877976 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:11.095671 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 553/1107] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:16:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:16:11.101153 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c67a3-85c9-402f-8d1a-9e385fcc987b None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:11.101619 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c67a3-85c9-402f-8d1a-9e385fcc987b None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:11.101723 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c67a3-85c9-402f-8d1a-9e385fcc987b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:11.119214 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-bc1c67a3-85c9-402f-8d1a-9e385fcc987b None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:16:11.120367 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-bc1c67a3-85c9-402f-8d1a-9e385fcc987b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:16:11.183911 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 555/1108] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:16:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:16:11.455718 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141712ee-7c14-4e4f-a3c9-54478f877976 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:11.468254 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:11.567237 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141712ee-7c14-4e4f-a3c9-54478f877976 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:11.600817 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fccdd4-67c1-4c7e-9d86-92ecfd9aa4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:11.601461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fccdd4-67c1-4c7e-9d86-92ecfd9aa4f6 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:11.601736 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fccdd4-67c1-4c7e-9d86-92ecfd9aa4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:11.602004 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fccdd4-67c1-4c7e-9d86-92ecfd9aa4f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:11.603085 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fccdd4-67c1-4c7e-9d86-92ecfd9aa4f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:11.604487 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fccdd4-67c1-4c7e-9d86-92ecfd9aa4f6 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:11.606052 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 554/1109] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:16:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:11.611751 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c67a3-85c9-402f-8d1a-9e385fcc987b None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:11.640098 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:11.714501 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c67a3-85c9-402f-8d1a-9e385fcc987b None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:11.752043 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b82f2e8-4dba-416c-8aa5-0f790810e260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:11.752639 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82f2e8-4dba-416c-8aa5-0f790810e260 None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:11.752914 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82f2e8-4dba-416c-8aa5-0f790810e260 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:11.753182 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82f2e8-4dba-416c-8aa5-0f790810e260 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:11.754750 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b82f2e8-4dba-416c-8aa5-0f790810e260 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:11.756211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b82f2e8-4dba-416c-8aa5-0f790810e260 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:11.777687 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 556/1110] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Sep 9 20:16:11 2022] POST /identity/v3/users => generated 286 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:16:13.958400 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50fb18-02e7-430b-b112-d7260aaac726 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:13.958931 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50fb18-02e7-430b-b112-d7260aaac726 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:13.958931 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50fb18-02e7-430b-b112-d7260aaac726 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:13.960249 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 555/1111] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:16:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:16:13.965635 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf89b5-c9f8-4368-8f1e-628e213d6bef None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:13.966059 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf89b5-c9f8-4368-8f1e-628e213d6bef None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:13.966188 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf89b5-c9f8-4368-8f1e-628e213d6bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:13.984425 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-1baf89b5-c9f8-4368-8f1e-628e213d6bef None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:16:13.985653 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-1baf89b5-c9f8-4368-8f1e-628e213d6bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:16:14.056644 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 557/1112] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:16:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:16:14.359982 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df50fb18-02e7-430b-b112-d7260aaac726 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:14.438682 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:14.463675 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df50fb18-02e7-430b-b112-d7260aaac726 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:14.504423 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60779e7f-e3e9-405c-aea3-6866989a70f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:14.505202 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60779e7f-e3e9-405c-aea3-6866989a70f5 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:14.505596 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60779e7f-e3e9-405c-aea3-6866989a70f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:14.505963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60779e7f-e3e9-405c-aea3-6866989a70f5 None admin] PATH_INFO: `/v3/roles/d5528405b19b48a6bc4b7ff451332d8a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:14.507686 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60779e7f-e3e9-405c-aea3-6866989a70f5 None admin] RBAC: Authorizing `identity:get_role(role_id=d5528405b19b48a6bc4b7ff451332d8a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:14.509404 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60779e7f-e3e9-405c-aea3-6866989a70f5 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:14.511079 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 556/1113] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:16:14 2022] GET /identity/v3/roles/d5528405b19b48a6bc4b7ff451332d8a => generated 223 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:14.516936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baf89b5-c9f8-4368-8f1e-628e213d6bef None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:14.531298 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:14.615267 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baf89b5-c9f8-4368-8f1e-628e213d6bef None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:14.648251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3f05b-7665-422c-b4e8-50e29030a4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:14.648873 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3f05b-7665-422c-b4e8-50e29030a4c2 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:14.649151 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3f05b-7665-422c-b4e8-50e29030a4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:14.649456 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3f05b-7665-422c-b4e8-50e29030a4c2 None admin] PATH_INFO: `/v3/users/817c0a9665de492e8ddc702f4b5a7196` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:14.663241 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a3f05b-7665-422c-b4e8-50e29030a4c2 None admin] RBAC: Authorizing `identity:get_user(user_id=817c0a9665de492e8ddc702f4b5a7196)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:14.664666 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a3f05b-7665-422c-b4e8-50e29030a4c2 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:14.666301 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 558/1114] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:16:14 2022] GET /identity/v3/users/817c0a9665de492e8ddc702f4b5a7196 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:16:14.672303 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60779e7f-e3e9-405c-aea3-6866989a70f5 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:14.769962 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60779e7f-e3e9-405c-aea3-6866989a70f5 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:14.802966 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9476523-0ff0-49b1-bc06-3f486f54dd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:14.803567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9476523-0ff0-49b1-bc06-3f486f54dd85 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:14.803848 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9476523-0ff0-49b1-bc06-3f486f54dd85 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:14.804126 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9476523-0ff0-49b1-bc06-3f486f54dd85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:14.818660 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:14.869694 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9476523-0ff0-49b1-bc06-3f486f54dd85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:14.871056 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9476523-0ff0-49b1-bc06-3f486f54dd85 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:14.964846 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 557/1115] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:16:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:16:14.970903 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3f05b-7665-422c-b4e8-50e29030a4c2 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:15.047824 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:15.079601 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3f05b-7665-422c-b4e8-50e29030a4c2 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:15.113725 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506762a3-11bc-4d1d-9089-c4dbf800b526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:15.114347 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506762a3-11bc-4d1d-9089-c4dbf800b526 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:15.114625 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506762a3-11bc-4d1d-9089-c4dbf800b526 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:15.114896 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506762a3-11bc-4d1d-9089-c4dbf800b526 None admin] PATH_INFO: `/v3/projects/81d44cfe705d437987b23852c51e07cc` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:15.115999 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506762a3-11bc-4d1d-9089-c4dbf800b526 None admin] RBAC: Authorizing `identity:get_project(project_id=81d44cfe705d437987b23852c51e07cc)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:15.117452 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506762a3-11bc-4d1d-9089-c4dbf800b526 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:15.119072 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 559/1116] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:16:14 2022] GET /identity/v3/projects/81d44cfe705d437987b23852c51e07cc => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:15.125435 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9476523-0ff0-49b1-bc06-3f486f54dd85 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:15.170265 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:15.220208 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9476523-0ff0-49b1-bc06-3f486f54dd85 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:15.252350 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8c9e7-efb6-4fe8-b1bd-e611ef8b7765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:15.252936 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8c9e7-efb6-4fe8-b1bd-e611ef8b7765 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:15.253211 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8c9e7-efb6-4fe8-b1bd-e611ef8b7765 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:15.253799 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8c9e7-efb6-4fe8-b1bd-e611ef8b7765 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:15.254506 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c8c9e7-efb6-4fe8-b1bd-e611ef8b7765 None admin] RBAC: Adding query filter params (role.id=d5528405b19b48a6bc4b7ff451332d8a, scope.project.id=81d44cfe705d437987b23852c51e07cc, user.id=817c0a9665de492e8ddc702f4b5a7196) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:16:15.254806 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c8c9e7-efb6-4fe8-b1bd-e611ef8b7765 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:15.256190 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c8c9e7-efb6-4fe8-b1bd-e611ef8b7765 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:15.261274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 558/1117] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:16:15 2022] GET /identity/v3/role_assignments?user.id=817c0a9665de492e8ddc702f4b5a7196&scope.project.id=81d44cfe705d437987b23852c51e07cc&role.id=d5528405b19b48a6bc4b7ff451332d8a => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:16:17.343381 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043493f1-e08e-4ed5-968c-0f7839f90cf9 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:17.343936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043493f1-e08e-4ed5-968c-0f7839f90cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:17.343936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043493f1-e08e-4ed5-968c-0f7839f90cf9 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:17.345269 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 560/1118] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:16:17.350381 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419687e-18e2-4fd9-8ea4-82c978c47c1a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:17.350796 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419687e-18e2-4fd9-8ea4-82c978c47c1a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:17.350965 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419687e-18e2-4fd9-8ea4-82c978c47c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:17.368631 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-b419687e-18e2-4fd9-8ea4-82c978c47c1a None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:16:17.369845 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b419687e-18e2-4fd9-8ea4-82c978c47c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:16:17.434674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 559/1119] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:16:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:16:17.700360 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043493f1-e08e-4ed5-968c-0f7839f90cf9 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:17.799912 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043493f1-e08e-4ed5-968c-0f7839f90cf9 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:17.835055 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29fd5c-16ec-406d-aeba-f905cc70ed77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:17.835675 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29fd5c-16ec-406d-aeba-f905cc70ed77 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:17.835954 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29fd5c-16ec-406d-aeba-f905cc70ed77 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:17.836228 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29fd5c-16ec-406d-aeba-f905cc70ed77 None admin] PATH_INFO: `/v3/roles/d5528405b19b48a6bc4b7ff451332d8a` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:17.837646 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f29fd5c-16ec-406d-aeba-f905cc70ed77 None admin] RBAC: Authorizing `identity:get_role(role_id=d5528405b19b48a6bc4b7ff451332d8a)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:17.839010 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f29fd5c-16ec-406d-aeba-f905cc70ed77 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:17.840322 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 561/1120] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:16:17 2022] GET /identity/v3/roles/d5528405b19b48a6bc4b7ff451332d8a => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:17.846127 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b419687e-18e2-4fd9-8ea4-82c978c47c1a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:17.886870 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:17.943285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b419687e-18e2-4fd9-8ea4-82c978c47c1a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:17.975682 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5032056b-c8c0-4fdc-8e9a-00ad7437b547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:17.976303 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032056b-c8c0-4fdc-8e9a-00ad7437b547 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:17.976583 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032056b-c8c0-4fdc-8e9a-00ad7437b547 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:17.976854 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032056b-c8c0-4fdc-8e9a-00ad7437b547 None admin] PATH_INFO: `/v3/users/817c0a9665de492e8ddc702f4b5a7196` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:17.977971 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5032056b-c8c0-4fdc-8e9a-00ad7437b547 None admin] RBAC: Authorizing `identity:get_user(user_id=817c0a9665de492e8ddc702f4b5a7196)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:17.979377 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5032056b-c8c0-4fdc-8e9a-00ad7437b547 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:17.980926 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 560/1121] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:16:17 2022] GET /identity/v3/users/817c0a9665de492e8ddc702f4b5a7196 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:17.986620 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29fd5c-16ec-406d-aeba-f905cc70ed77 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:17.995793 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:18.084834 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29fd5c-16ec-406d-aeba-f905cc70ed77 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:18.117676 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d7059b-b748-4a8f-9d84-c464af87f5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:18.118293 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7059b-b748-4a8f-9d84-c464af87f5f7 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:18.118570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7059b-b748-4a8f-9d84-c464af87f5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:18.118840 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7059b-b748-4a8f-9d84-c464af87f5f7 None admin] PATH_INFO: `/v3/projects/81d44cfe705d437987b23852c51e07cc` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:18.119928 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d7059b-b748-4a8f-9d84-c464af87f5f7 None admin] RBAC: Authorizing `identity:get_project(project_id=81d44cfe705d437987b23852c51e07cc)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:18.121370 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d7059b-b748-4a8f-9d84-c464af87f5f7 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:18.122963 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 562/1122] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:16:17 2022] GET /identity/v3/projects/81d44cfe705d437987b23852c51e07cc => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:18.129237 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5032056b-c8c0-4fdc-8e9a-00ad7437b547 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:18.224497 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5032056b-c8c0-4fdc-8e9a-00ad7437b547 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:18.232899 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:18.257839 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c348513-7e8b-4ae7-818c-e4e78653dad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:18.258486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c348513-7e8b-4ae7-818c-e4e78653dad9 None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:18.258770 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c348513-7e8b-4ae7-818c-e4e78653dad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:18.259050 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c348513-7e8b-4ae7-818c-e4e78653dad9 None admin] PATH_INFO: `/v3/projects/81d44cfe705d437987b23852c51e07cc/users/817c0a9665de492e8ddc702f4b5a7196/roles/d5528405b19b48a6bc4b7ff451332d8a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:18.261159 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c348513-7e8b-4ae7-818c-e4e78653dad9 None admin] RBAC: Authorizing `identity:create_grant(project_id=81d44cfe705d437987b23852c51e07cc, user_id=817c0a9665de492e8ddc702f4b5a7196, role_id=d5528405b19b48a6bc4b7ff451332d8a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:18.262587 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c348513-7e8b-4ae7-818c-e4e78653dad9 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:18.274890 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 561/1123] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Sep 9 20:16:18 2022] PUT /identity/v3/projects/81d44cfe705d437987b23852c51e07cc/users/817c0a9665de492e8ddc702f4b5a7196/roles/d5528405b19b48a6bc4b7ff451332d8a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:16:20.382362 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0200b1-c866-40de-a878-4d60bab85fc5 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:20.382892 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0200b1-c866-40de-a878-4d60bab85fc5 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:20.382892 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0200b1-c866-40de-a878-4d60bab85fc5 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:20.384246 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 563/1124] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:16:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:16:20.389345 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd1039-df91-4bfb-b136-34479540b18d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:20.389749 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd1039-df91-4bfb-b136-34479540b18d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:20.389900 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd1039-df91-4bfb-b136-34479540b18d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:20.408064 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-47fd1039-df91-4bfb-b136-34479540b18d None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:16:20.409236 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-47fd1039-df91-4bfb-b136-34479540b18d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:16:20.473477 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 562/1125] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:16:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:16:20.746030 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0200b1-c866-40de-a878-4d60bab85fc5 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:20.824532 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:20.852710 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0200b1-c866-40de-a878-4d60bab85fc5 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:20.888538 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246e9ed-8d22-4c73-9a15-dd685148af3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:20.889158 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246e9ed-8d22-4c73-9a15-dd685148af3e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:20.889462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246e9ed-8d22-4c73-9a15-dd685148af3e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:20.889731 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246e9ed-8d22-4c73-9a15-dd685148af3e None admin] PATH_INFO: `/v3/roles/d5528405b19b48a6bc4b7ff451332d8a` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:20.891085 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5246e9ed-8d22-4c73-9a15-dd685148af3e None admin] RBAC: Authorizing `identity:get_role(role_id=d5528405b19b48a6bc4b7ff451332d8a)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:20.892433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5246e9ed-8d22-4c73-9a15-dd685148af3e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:20.893764 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 564/1126] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:16:20 2022] GET /identity/v3/roles/d5528405b19b48a6bc4b7ff451332d8a => generated 223 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:20.900412 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd1039-df91-4bfb-b136-34479540b18d None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:20.991909 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:20.995703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd1039-df91-4bfb-b136-34479540b18d None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:21.029885 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97562cc0-ed13-4bb1-9e68-b1a225a6e5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:21.030529 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97562cc0-ed13-4bb1-9e68-b1a225a6e5b4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:21.030807 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97562cc0-ed13-4bb1-9e68-b1a225a6e5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:21.031075 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97562cc0-ed13-4bb1-9e68-b1a225a6e5b4 None admin] PATH_INFO: `/v3/users/817c0a9665de492e8ddc702f4b5a7196` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:21.032158 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97562cc0-ed13-4bb1-9e68-b1a225a6e5b4 None admin] RBAC: Authorizing `identity:get_user(user_id=817c0a9665de492e8ddc702f4b5a7196)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:21.033588 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97562cc0-ed13-4bb1-9e68-b1a225a6e5b4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:21.035152 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 563/1127] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:16:20 2022] GET /identity/v3/users/817c0a9665de492e8ddc702f4b5a7196 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:21.041124 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246e9ed-8d22-4c73-9a15-dd685148af3e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:21.138085 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246e9ed-8d22-4c73-9a15-dd685148af3e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:21.170254 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898b698-9c02-4dbf-aa8f-951de5fbbf2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:21.170844 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898b698-9c02-4dbf-aa8f-951de5fbbf2c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:21.171123 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898b698-9c02-4dbf-aa8f-951de5fbbf2c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:21.171403 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898b698-9c02-4dbf-aa8f-951de5fbbf2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:21.183172 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:21.237193 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5898b698-9c02-4dbf-aa8f-951de5fbbf2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:21.239290 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5898b698-9c02-4dbf-aa8f-951de5fbbf2c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:21.336293 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 565/1128] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:16:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:16:21.342167 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97562cc0-ed13-4bb1-9e68-b1a225a6e5b4 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:21.439504 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97562cc0-ed13-4bb1-9e68-b1a225a6e5b4 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:21.473305 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157e605-627e-45cf-b1da-5dbf634ef2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:21.473921 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157e605-627e-45cf-b1da-5dbf634ef2c2 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:21.474198 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157e605-627e-45cf-b1da-5dbf634ef2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:21.474471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157e605-627e-45cf-b1da-5dbf634ef2c2 None admin] PATH_INFO: `/v3/projects/81d44cfe705d437987b23852c51e07cc` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:21.475582 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8157e605-627e-45cf-b1da-5dbf634ef2c2 None admin] RBAC: Authorizing `identity:get_project(project_id=81d44cfe705d437987b23852c51e07cc)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:21.477080 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8157e605-627e-45cf-b1da-5dbf634ef2c2 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:21.478771 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 564/1129] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:16:21 2022] GET /identity/v3/projects/81d44cfe705d437987b23852c51e07cc => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:21.484763 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898b698-9c02-4dbf-aa8f-951de5fbbf2c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:21.526772 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:21.581865 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898b698-9c02-4dbf-aa8f-951de5fbbf2c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:21.615156 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0ae98-6d3a-4c69-975c-e4e2915ee847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:21.615692 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0ae98-6d3a-4c69-975c-e4e2915ee847 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:21.615971 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0ae98-6d3a-4c69-975c-e4e2915ee847 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:21.616243 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0ae98-6d3a-4c69-975c-e4e2915ee847 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:21.616959 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff0ae98-6d3a-4c69-975c-e4e2915ee847 None admin] RBAC: Adding query filter params (role.id=d5528405b19b48a6bc4b7ff451332d8a, scope.project.id=81d44cfe705d437987b23852c51e07cc, user.id=817c0a9665de492e8ddc702f4b5a7196) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:16:21.617286 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff0ae98-6d3a-4c69-975c-e4e2915ee847 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:21.618707 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff0ae98-6d3a-4c69-975c-e4e2915ee847 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:21.624288 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 566/1130] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:16:21 2022] GET /identity/v3/role_assignments?user.id=817c0a9665de492e8ddc702f4b5a7196&scope.project.id=81d44cfe705d437987b23852c51e07cc&role.id=d5528405b19b48a6bc4b7ff451332d8a => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:23.763998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bccec-2620-490f-b6d2-6dd324acf531 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:23.764500 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bccec-2620-490f-b6d2-6dd324acf531 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:23.764585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bccec-2620-490f-b6d2-6dd324acf531 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:23.765896 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 565/1131] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:16:23.771183 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc56a0f-204d-4628-af48-1a65e83e840f None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:23.771580 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc56a0f-204d-4628-af48-1a65e83e840f None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:23.771735 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc56a0f-204d-4628-af48-1a65e83e840f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:23.789920 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-3fc56a0f-204d-4628-af48-1a65e83e840f None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:16:23.791085 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-3fc56a0f-204d-4628-af48-1a65e83e840f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:16:23.858905 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 567/1132] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:16:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:16:24.136595 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708bccec-2620-490f-b6d2-6dd324acf531 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:24.172557 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:24.241446 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708bccec-2620-490f-b6d2-6dd324acf531 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:24.274887 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b26c97-beff-4648-adc0-b4173a6b8daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:24.275506 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b26c97-beff-4648-adc0-b4173a6b8daf None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:24.275786 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b26c97-beff-4648-adc0-b4173a6b8daf None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:24.276060 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b26c97-beff-4648-adc0-b4173a6b8daf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:24.277159 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b26c97-beff-4648-adc0-b4173a6b8daf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:24.278774 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b26c97-beff-4648-adc0-b4173a6b8daf None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:24.280397 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 566/1133] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:16:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:24.286708 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc56a0f-204d-4628-af48-1a65e83e840f None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:24.319211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:24.381519 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc56a0f-204d-4628-af48-1a65e83e840f None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:24.414855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a54212-59ba-4e38-afba-959c7ffa4aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:24.415393 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a54212-59ba-4e38-afba-959c7ffa4aba None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:24.415670 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a54212-59ba-4e38-afba-959c7ffa4aba None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:24.415943 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a54212-59ba-4e38-afba-959c7ffa4aba None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:24.416577 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a54212-59ba-4e38-afba-959c7ffa4aba None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:24.418012 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a54212-59ba-4e38-afba-959c7ffa4aba None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:24.443421 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 568/1134] 10.222.0.26 () {60 vars in 1141 bytes} [Fri Sep 9 20:16:24 2022] POST /identity/v3/projects => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:16:26.618730 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6755b4b-ede3-43db-999c-d8e6e5b9ad80 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:26.619291 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6755b4b-ede3-43db-999c-d8e6e5b9ad80 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:26.619291 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6755b4b-ede3-43db-999c-d8e6e5b9ad80 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:26.620583 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 567/1135] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:16:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:16:26.625715 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bfed2f-4e90-4ce1-9e43-9779df0b6fcf None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:26.626160 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bfed2f-4e90-4ce1-9e43-9779df0b6fcf None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:26.626267 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bfed2f-4e90-4ce1-9e43-9779df0b6fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:26.644734 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-b8bfed2f-4e90-4ce1-9e43-9779df0b6fcf None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:16:26.645933 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-b8bfed2f-4e90-4ce1-9e43-9779df0b6fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:16:26.709938 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 569/1136] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:16:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:16:26.975073 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6755b4b-ede3-43db-999c-d8e6e5b9ad80 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:27.072989 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6755b4b-ede3-43db-999c-d8e6e5b9ad80 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:27.074487 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:27.107102 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3010cc0-cd96-4b3e-ae40-5edb00adc9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:27.107721 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3010cc0-cd96-4b3e-ae40-5edb00adc9c5 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:27.108002 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3010cc0-cd96-4b3e-ae40-5edb00adc9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:27.108274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3010cc0-cd96-4b3e-ae40-5edb00adc9c5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:27.109405 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3010cc0-cd96-4b3e-ae40-5edb00adc9c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:27.110810 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3010cc0-cd96-4b3e-ae40-5edb00adc9c5 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:27.112354 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 568/1137] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:16:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:27.118470 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bfed2f-4e90-4ce1-9e43-9779df0b6fcf None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:27.136401 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:27.213635 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bfed2f-4e90-4ce1-9e43-9779df0b6fcf None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:27.246112 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64709d19-8e19-4b47-91c6-fd185dcaa51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:27.246653 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64709d19-8e19-4b47-91c6-fd185dcaa51c None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:27.246930 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64709d19-8e19-4b47-91c6-fd185dcaa51c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:27.247203 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64709d19-8e19-4b47-91c6-fd185dcaa51c None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:27.247851 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64709d19-8e19-4b47-91c6-fd185dcaa51c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:27.249302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64709d19-8e19-4b47-91c6-fd185dcaa51c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:27.270190 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 570/1138] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Sep 9 20:16:27 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:16:29.456842 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a580c-5bb5-4e23-b771-0250195149cf None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:29.457515 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a580c-5bb5-4e23-b771-0250195149cf None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:29.457515 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a580c-5bb5-4e23-b771-0250195149cf None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:29.458766 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 569/1139] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:16:29.464035 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5bb1f-9856-423e-920e-4bd27400fe81 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:29.464476 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5bb1f-9856-423e-920e-4bd27400fe81 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:29.464571 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5bb1f-9856-423e-920e-4bd27400fe81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:29.483159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-8af5bb1f-9856-423e-920e-4bd27400fe81 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:16:29.484313 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-8af5bb1f-9856-423e-920e-4bd27400fe81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:16:29.547468 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 571/1140] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:16:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:16:29.812643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a580c-5bb5-4e23-b771-0250195149cf None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:29.913858 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a580c-5bb5-4e23-b771-0250195149cf None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:29.946620 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d17a1-1192-4469-acb3-54f827273b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:29.947238 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d17a1-1192-4469-acb3-54f827273b59 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:29.947515 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d17a1-1192-4469-acb3-54f827273b59 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:29.947783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d17a1-1192-4469-acb3-54f827273b59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:29.957428 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8d17a1-1192-4469-acb3-54f827273b59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:16:29.962128 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-0d8d17a1-1192-4469-acb3-54f827273b59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:16:29.962128 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:16:29.962128 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:16:29.962128 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:16:29.962128 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:16:29.962128 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:16:29.962128 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:16:29.962128 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:16:29.962128 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:16:29.962128 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:16:29.962128 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:16:29.962128 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:16:29.962128 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:16:29.962128 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:16:29.962128 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:16:29.962128 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:16:29.963258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:16:29.963258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:16:29.963258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:16:29.963258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:16:29.963258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:16:29.963258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:16:29.963258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:16:29.963258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:16:29.963258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:16:29.963258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:16:29.963258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:16:29.963258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:16:29.963258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:16:29.963258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:16:29.963258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:16:29.963258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:16:29.964519 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:16:29.964519 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:16:29.964519 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:16:29.964519 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:16:29.964519 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:16:29.964519 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:16:29.964519 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:16:29.964519 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:16:29.964519 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:16:29.964519 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:16:29.964519 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:16:29.964519 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:16:29.964519 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 570/1141] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:16:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:16:29.969300 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5bb1f-9856-423e-920e-4bd27400fe81 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:29.972943 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:30.066282 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5bb1f-9856-423e-920e-4bd27400fe81 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:30.098668 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda46a7-fa57-461d-bac4-3c387ea4af3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:30.099268 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda46a7-fa57-461d-bac4-3c387ea4af3e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:30.099546 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda46a7-fa57-461d-bac4-3c387ea4af3e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:30.099813 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda46a7-fa57-461d-bac4-3c387ea4af3e None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:30.100468 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dda46a7-fa57-461d-bac4-3c387ea4af3e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:16:30.100766 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dda46a7-fa57-461d-bac4-3c387ea4af3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:30.102144 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dda46a7-fa57-461d-bac4-3c387ea4af3e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:30.109884 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 572/1142] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:16:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:30.116421 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d17a1-1192-4469-acb3-54f827273b59 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:30.146532 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:30.213326 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d17a1-1192-4469-acb3-54f827273b59 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:30.245340 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065fafc4-c3af-4380-aecc-839fbdab25bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:30.245948 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065fafc4-c3af-4380-aecc-839fbdab25bc None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:30.246222 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065fafc4-c3af-4380-aecc-839fbdab25bc None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:30.246490 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065fafc4-c3af-4380-aecc-839fbdab25bc None admin] PATH_INFO: `/v3/users/5a3ebbb190e64b2d9f4425b80946eec0` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:30.260032 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065fafc4-c3af-4380-aecc-839fbdab25bc None admin] RBAC: Authorizing `identity:get_user(user_id=5a3ebbb190e64b2d9f4425b80946eec0)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:30.261457 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065fafc4-c3af-4380-aecc-839fbdab25bc None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:30.263001 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 571/1143] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:16:30 2022] GET /identity/v3/users/5a3ebbb190e64b2d9f4425b80946eec0 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:30.268773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda46a7-fa57-461d-bac4-3c387ea4af3e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:30.331674 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:30.365287 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda46a7-fa57-461d-bac4-3c387ea4af3e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:30.405641 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb86f8-e7d0-4c9f-bb63-3cbc26b99f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:30.406246 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb86f8-e7d0-4c9f-bb63-3cbc26b99f39 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:30.406526 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb86f8-e7d0-4c9f-bb63-3cbc26b99f39 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:30.406799 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb86f8-e7d0-4c9f-bb63-3cbc26b99f39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:30.472306 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfb86f8-e7d0-4c9f-bb63-3cbc26b99f39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:30.473771 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfb86f8-e7d0-4c9f-bb63-3cbc26b99f39 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:30.534836 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:30.570231 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 573/1144] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:16:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:16:30.576133 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065fafc4-c3af-4380-aecc-839fbdab25bc None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:30.661225 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:30.673427 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065fafc4-c3af-4380-aecc-839fbdab25bc None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:30.706553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a9f85-d6cd-4122-9dc7-41d80cf5231a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:30.707203 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a9f85-d6cd-4122-9dc7-41d80cf5231a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:30.707485 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a9f85-d6cd-4122-9dc7-41d80cf5231a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:30.707759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a9f85-d6cd-4122-9dc7-41d80cf5231a None admin] PATH_INFO: `/v3/projects/a4f699cae84847a4a0193b4d1bea1f59` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:30.708872 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a9f85-d6cd-4122-9dc7-41d80cf5231a None admin] RBAC: Authorizing `identity:get_project(project_id=a4f699cae84847a4a0193b4d1bea1f59)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:30.710308 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a9f85-d6cd-4122-9dc7-41d80cf5231a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:30.711929 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 572/1145] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:16:30 2022] GET /identity/v3/projects/a4f699cae84847a4a0193b4d1bea1f59 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:30.718007 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb86f8-e7d0-4c9f-bb63-3cbc26b99f39 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:30.813828 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb86f8-e7d0-4c9f-bb63-3cbc26b99f39 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:30.845797 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb5d181-3596-471e-9c4b-aca09ad184e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:30.846385 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5d181-3596-471e-9c4b-aca09ad184e2 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:30.846662 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5d181-3596-471e-9c4b-aca09ad184e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:30.846929 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5d181-3596-471e-9c4b-aca09ad184e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:30.848170 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb5d181-3596-471e-9c4b-aca09ad184e2 None admin] RBAC: Adding query filter params (role.id=9121ba25a6d84756990f0c3a19e9fecb, scope.project.id=a4f699cae84847a4a0193b4d1bea1f59, user.id=5a3ebbb190e64b2d9f4425b80946eec0) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:16:30.848473 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb5d181-3596-471e-9c4b-aca09ad184e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:30.849907 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb5d181-3596-471e-9c4b-aca09ad184e2 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:30.855027 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 574/1146] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:16:30 2022] GET /identity/v3/role_assignments?user.id=5a3ebbb190e64b2d9f4425b80946eec0&scope.project.id=a4f699cae84847a4a0193b4d1bea1f59&role.id=9121ba25a6d84756990f0c3a19e9fecb => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:32.976515 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921723a-625d-4fea-a34b-0ca90f9d54d6 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:32.977051 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921723a-625d-4fea-a34b-0ca90f9d54d6 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:32.977051 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921723a-625d-4fea-a34b-0ca90f9d54d6 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:32.978407 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 573/1147] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:16:32.983485 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139c937-88d5-4b0c-89b1-3717b596f6da None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:32.983908 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139c937-88d5-4b0c-89b1-3717b596f6da None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:32.984026 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139c937-88d5-4b0c-89b1-3717b596f6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:33.001776 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-6139c937-88d5-4b0c-89b1-3717b596f6da None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:16:33.003055 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6139c937-88d5-4b0c-89b1-3717b596f6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:16:33.027099 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:33.069760 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 575/1148] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:16:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:16:33.338912 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6921723a-625d-4fea-a34b-0ca90f9d54d6 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:33.436969 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6921723a-625d-4fea-a34b-0ca90f9d54d6 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:33.469230 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd08ddde-12fa-45a9-9eb6-83053dc7d403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:33.469860 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08ddde-12fa-45a9-9eb6-83053dc7d403 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:33.470137 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08ddde-12fa-45a9-9eb6-83053dc7d403 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:33.470406 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08ddde-12fa-45a9-9eb6-83053dc7d403 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:33.480007 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd08ddde-12fa-45a9-9eb6-83053dc7d403 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:16:33.484710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-cd08ddde-12fa-45a9-9eb6-83053dc7d403 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:16:33.484710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:16:33.484710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:16:33.484710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:16:33.484710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:16:33.484710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:16:33.484710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:16:33.484710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:16:33.484710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:16:33.484710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:16:33.484710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:16:33.484710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:16:33.484710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:16:33.484710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:16:33.484710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:16:33.484710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:16:33.485863 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:16:33.485863 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:16:33.485863 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:16:33.485863 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:16:33.485863 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:16:33.485863 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:16:33.485863 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:16:33.485863 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:16:33.485863 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:16:33.485863 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:16:33.485863 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:16:33.485863 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:16:33.485863 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:16:33.485863 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:16:33.485863 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:16:33.485863 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:16:33.486921 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:16:33.486921 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:16:33.486921 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:16:33.486921 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:16:33.486921 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:16:33.486921 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:16:33.486921 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:16:33.486921 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:16:33.486921 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:16:33.486921 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:16:33.486921 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:16:33.486921 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:16:33.486921 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 574/1149] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:16:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:16:33.492322 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6139c937-88d5-4b0c-89b1-3717b596f6da None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:33.588002 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6139c937-88d5-4b0c-89b1-3717b596f6da None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:33.621621 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facbc02c-59c9-4626-a920-35b494efb801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:33.622233 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbc02c-59c9-4626-a920-35b494efb801 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:33.622541 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbc02c-59c9-4626-a920-35b494efb801 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:33.622809 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbc02c-59c9-4626-a920-35b494efb801 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:33.623459 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facbc02c-59c9-4626-a920-35b494efb801 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:16:33.623754 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facbc02c-59c9-4626-a920-35b494efb801 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:33.625105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facbc02c-59c9-4626-a920-35b494efb801 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:33.633159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 576/1150] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:16:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:33.638944 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd08ddde-12fa-45a9-9eb6-83053dc7d403 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:33.657179 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:33.736556 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd08ddde-12fa-45a9-9eb6-83053dc7d403 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:33.769390 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c2b4f-ed90-405e-bfaf-7933a412da62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:33.770006 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c2b4f-ed90-405e-bfaf-7933a412da62 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:33.770282 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c2b4f-ed90-405e-bfaf-7933a412da62 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:33.770552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c2b4f-ed90-405e-bfaf-7933a412da62 None admin] PATH_INFO: `/v3/users/5a3ebbb190e64b2d9f4425b80946eec0` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:33.771638 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5c2b4f-ed90-405e-bfaf-7933a412da62 None admin] RBAC: Authorizing `identity:get_user(user_id=5a3ebbb190e64b2d9f4425b80946eec0)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:33.773149 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5c2b4f-ed90-405e-bfaf-7933a412da62 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:33.774865 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 575/1151] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:16:33 2022] GET /identity/v3/users/5a3ebbb190e64b2d9f4425b80946eec0 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:33.780674 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facbc02c-59c9-4626-a920-35b494efb801 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:33.806570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:33.881601 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facbc02c-59c9-4626-a920-35b494efb801 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:33.916143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4eb14-5fed-4310-9012-3e3e0a8cafb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:33.916759 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4eb14-5fed-4310-9012-3e3e0a8cafb9 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:33.917034 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4eb14-5fed-4310-9012-3e3e0a8cafb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:33.917331 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4eb14-5fed-4310-9012-3e3e0a8cafb9 None admin] PATH_INFO: `/v3/projects/a4f699cae84847a4a0193b4d1bea1f59` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:33.918415 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4eb14-5fed-4310-9012-3e3e0a8cafb9 None admin] RBAC: Authorizing `identity:get_project(project_id=a4f699cae84847a4a0193b4d1bea1f59)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:33.919826 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4eb14-5fed-4310-9012-3e3e0a8cafb9 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:33.921452 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 577/1152] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:16:33 2022] GET /identity/v3/projects/a4f699cae84847a4a0193b4d1bea1f59 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:33.927541 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c2b4f-ed90-405e-bfaf-7933a412da62 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:34.008996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:34.025529 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c2b4f-ed90-405e-bfaf-7933a412da62 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:34.058342 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ca882-94e6-48da-93d3-a91b5870d268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:34.058996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ca882-94e6-48da-93d3-a91b5870d268 None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:34.059274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ca882-94e6-48da-93d3-a91b5870d268 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:34.059544 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ca882-94e6-48da-93d3-a91b5870d268 None admin] PATH_INFO: `/v3/projects/a4f699cae84847a4a0193b4d1bea1f59/users/5a3ebbb190e64b2d9f4425b80946eec0/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:34.061437 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ca882-94e6-48da-93d3-a91b5870d268 None admin] RBAC: Authorizing `identity:create_grant(project_id=a4f699cae84847a4a0193b4d1bea1f59, user_id=5a3ebbb190e64b2d9f4425b80946eec0, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:34.062841 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ca882-94e6-48da-93d3-a91b5870d268 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:34.076036 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 576/1153] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Sep 9 20:16:33 2022] PUT /identity/v3/projects/a4f699cae84847a4a0193b4d1bea1f59/users/5a3ebbb190e64b2d9f4425b80946eec0/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:16:36.192812 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ad6db-1406-4729-ab6e-aebb76879046 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:36.193416 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ad6db-1406-4729-ab6e-aebb76879046 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:36.193517 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ad6db-1406-4729-ab6e-aebb76879046 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:36.194862 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 578/1154] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:16:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:16:36.200154 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1249a90-f02c-4f17-a9f1-7fc7cba8b4f0 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:36.200594 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1249a90-f02c-4f17-a9f1-7fc7cba8b4f0 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:36.200696 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1249a90-f02c-4f17-a9f1-7fc7cba8b4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:36.218694 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a1249a90-f02c-4f17-a9f1-7fc7cba8b4f0 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:16:36.219886 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a1249a90-f02c-4f17-a9f1-7fc7cba8b4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:16:36.284140 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 577/1155] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:16:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:16:36.551156 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ad6db-1406-4729-ab6e-aebb76879046 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:36.638269 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:36.648544 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ad6db-1406-4729-ab6e-aebb76879046 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:36.682040 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66637cb2-0f46-48db-a88e-8555f6993c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:36.682660 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66637cb2-0f46-48db-a88e-8555f6993c8a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:36.682935 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66637cb2-0f46-48db-a88e-8555f6993c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:36.683204 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66637cb2-0f46-48db-a88e-8555f6993c8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:36.693426 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66637cb2-0f46-48db-a88e-8555f6993c8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:16:36.698131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-66637cb2-0f46-48db-a88e-8555f6993c8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:16:36.698131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:16:36.698131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:16:36.698131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:16:36.698131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:16:36.698131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:16:36.698131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:16:36.698131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:16:36.698131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:16:36.698131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:16:36.698131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:16:36.698131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:16:36.698131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:16:36.698131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:16:36.698131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:16:36.698131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:16:36.699337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:16:36.699337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:16:36.699337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:16:36.699337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:16:36.699337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:16:36.699337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:16:36.699337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:16:36.699337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:16:36.699337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:16:36.699337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:16:36.699337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:16:36.699337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:16:36.699337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:16:36.699337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:16:36.699337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:16:36.699337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:16:36.700433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:16:36.700433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:16:36.700433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:16:36.700433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:16:36.700433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:16:36.700433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:16:36.700433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:16:36.700433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:16:36.700433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:16:36.700433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:16:36.700433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:16:36.700433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:16:36.700433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 579/1156] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:16:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:16:36.705124 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1249a90-f02c-4f17-a9f1-7fc7cba8b4f0 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:36.776781 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:36.802438 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1249a90-f02c-4f17-a9f1-7fc7cba8b4f0 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:36.834137 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bbf8bf-b8d3-4f4d-b042-e4c920c07853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:36.834734 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bbf8bf-b8d3-4f4d-b042-e4c920c07853 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:36.835017 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bbf8bf-b8d3-4f4d-b042-e4c920c07853 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:36.835292 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bbf8bf-b8d3-4f4d-b042-e4c920c07853 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:36.835948 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bbf8bf-b8d3-4f4d-b042-e4c920c07853 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:16:36.836250 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bbf8bf-b8d3-4f4d-b042-e4c920c07853 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:36.837623 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bbf8bf-b8d3-4f4d-b042-e4c920c07853 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:36.845351 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 578/1157] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:16:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:16:36.851780 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66637cb2-0f46-48db-a88e-8555f6993c8a None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:36.948395 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66637cb2-0f46-48db-a88e-8555f6993c8a None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:36.981300 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d6b73-3152-4af7-a2ad-9cf22a23a715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:36.981918 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d6b73-3152-4af7-a2ad-9cf22a23a715 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:36.982201 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d6b73-3152-4af7-a2ad-9cf22a23a715 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:36.982476 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d6b73-3152-4af7-a2ad-9cf22a23a715 None admin] PATH_INFO: `/v3/users/5a3ebbb190e64b2d9f4425b80946eec0` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:36.983582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d6b73-3152-4af7-a2ad-9cf22a23a715 None admin] RBAC: Authorizing `identity:get_user(user_id=5a3ebbb190e64b2d9f4425b80946eec0)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:36.985001 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d6b73-3152-4af7-a2ad-9cf22a23a715 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:36.986577 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 580/1158] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:16:36 2022] GET /identity/v3/users/5a3ebbb190e64b2d9f4425b80946eec0 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:36.992227 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bbf8bf-b8d3-4f4d-b042-e4c920c07853 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:37.089121 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bbf8bf-b8d3-4f4d-b042-e4c920c07853 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:37.122261 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87484afd-300d-4dad-bef5-ee43c568b367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:37.122922 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87484afd-300d-4dad-bef5-ee43c568b367 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:37.123202 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87484afd-300d-4dad-bef5-ee43c568b367 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:37.123471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87484afd-300d-4dad-bef5-ee43c568b367 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:37.126914 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:37.188297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87484afd-300d-4dad-bef5-ee43c568b367 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:37.189748 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87484afd-300d-4dad-bef5-ee43c568b367 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:37.283218 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 579/1159] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:16:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:16:37.289293 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d6b73-3152-4af7-a2ad-9cf22a23a715 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:37.308128 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:37.386069 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d6b73-3152-4af7-a2ad-9cf22a23a715 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:37.418552 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84358a0b-2c16-4156-b825-b384266b97de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:37.419169 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84358a0b-2c16-4156-b825-b384266b97de None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:37.419445 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84358a0b-2c16-4156-b825-b384266b97de None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:37.419714 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84358a0b-2c16-4156-b825-b384266b97de None admin] PATH_INFO: `/v3/projects/a4f699cae84847a4a0193b4d1bea1f59` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:37.420793 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84358a0b-2c16-4156-b825-b384266b97de None admin] RBAC: Authorizing `identity:get_project(project_id=a4f699cae84847a4a0193b4d1bea1f59)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:37.422231 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84358a0b-2c16-4156-b825-b384266b97de None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:37.423883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 581/1160] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:16:37 2022] GET /identity/v3/projects/a4f699cae84847a4a0193b4d1bea1f59 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:37.429984 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87484afd-300d-4dad-bef5-ee43c568b367 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:37.463458 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:37.524968 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87484afd-300d-4dad-bef5-ee43c568b367 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:37.556617 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30560b4a-f713-4147-b1e7-ee3447202f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:37.557198 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30560b4a-f713-4147-b1e7-ee3447202f98 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:37.557507 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30560b4a-f713-4147-b1e7-ee3447202f98 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:37.557777 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30560b4a-f713-4147-b1e7-ee3447202f98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:37.558483 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30560b4a-f713-4147-b1e7-ee3447202f98 None admin] RBAC: Adding query filter params (role.id=9121ba25a6d84756990f0c3a19e9fecb, scope.project.id=a4f699cae84847a4a0193b4d1bea1f59, user.id=5a3ebbb190e64b2d9f4425b80946eec0) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:16:37.558786 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30560b4a-f713-4147-b1e7-ee3447202f98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:37.560179 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30560b4a-f713-4147-b1e7-ee3447202f98 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:37.565647 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 580/1161] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:16:37 2022] GET /identity/v3/role_assignments?user.id=5a3ebbb190e64b2d9f4425b80946eec0&scope.project.id=a4f699cae84847a4a0193b4d1bea1f59&role.id=9121ba25a6d84756990f0c3a19e9fecb => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:39.694126 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e42b33-71be-4a4d-ab4d-d56765ce7323 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:39.694531 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e42b33-71be-4a4d-ab4d-d56765ce7323 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:39.694664 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e42b33-71be-4a4d-ab4d-d56765ce7323 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:39.695997 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 582/1162] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:16:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:16:39.701270 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c03d1f-3ba6-4837-a242-9f747cd769c5 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:39.701716 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c03d1f-3ba6-4837-a242-9f747cd769c5 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:39.701818 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c03d1f-3ba6-4837-a242-9f747cd769c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:39.719429 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-31c03d1f-3ba6-4837-a242-9f747cd769c5 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:16:39.720600 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-31c03d1f-3ba6-4837-a242-9f747cd769c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:16:39.784703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 581/1163] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:16:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:16:40.052455 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e42b33-71be-4a4d-ab4d-d56765ce7323 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:40.136882 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:40.152983 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e42b33-71be-4a4d-ab4d-d56765ce7323 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:40.184987 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2451d-7e08-4bd8-956e-de882d1afa8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:40.185603 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2451d-7e08-4bd8-956e-de882d1afa8b None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:40.185877 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2451d-7e08-4bd8-956e-de882d1afa8b None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:40.186143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2451d-7e08-4bd8-956e-de882d1afa8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:40.249036 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae2451d-7e08-4bd8-956e-de882d1afa8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:40.250488 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae2451d-7e08-4bd8-956e-de882d1afa8b None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:40.329494 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:40.344849 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 583/1164] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:16:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:16:40.350836 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c03d1f-3ba6-4837-a242-9f747cd769c5 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:40.380151 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:40.448468 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c03d1f-3ba6-4837-a242-9f747cd769c5 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:40.487387 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687802e4-0f41-45b3-b12d-d856363183a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:40.488446 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687802e4-0f41-45b3-b12d-d856363183a3 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:40.488834 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687802e4-0f41-45b3-b12d-d856363183a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:40.489239 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687802e4-0f41-45b3-b12d-d856363183a3 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:40.496557 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687802e4-0f41-45b3-b12d-d856363183a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:40.498013 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687802e4-0f41-45b3-b12d-d856363183a3 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:40.509791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-687802e4-0f41-45b3-b12d-d856363183a3 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 09 20:16:40.509791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:16:40.509791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 09 20:16:40.509791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 09 20:16:40.509791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:16:40.509791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:16:40.509791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:16:40.509791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:16:40.509791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 09 20:16:40.509791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:16:40.509791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:16:40.509791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:16:40.509791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:16:40.509791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:16:40.509791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:16:40.509791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:16:40.509791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:16:40.511081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:16:40.511081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:16:40.511081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:16:40.511081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:16:40.511081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:16:40.511081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:16:40.511081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:16:40.511081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:16:40.511081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:16:40.511081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:16:40.511081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:16:40.511081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:16:40.511081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:16:40.511081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:16:40.511081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:16:40.511081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:16:40.512332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:16:40.512332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:16:40.512332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:16:40.512332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:16:40.512332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:16:40.512332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:16:40.512332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:16:40.512332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:16:40.512332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:16:40.512332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:16:40.512332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 09 20:16:40.512332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:16:40.512332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 09 20:16:40.512332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:16:40.512332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 582/1165] 10.222.0.26 () {56 vars in 1117 bytes} [Fri Sep 9 20:16:40 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:16:40.516988 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2451d-7e08-4bd8-956e-de882d1afa8b None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:40.611291 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2451d-7e08-4bd8-956e-de882d1afa8b None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:40.644041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d06542-6af9-4ea5-ad59-3331d37fde97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:40.644623 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d06542-6af9-4ea5-ad59-3331d37fde97 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:40.644897 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d06542-6af9-4ea5-ad59-3331d37fde97 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:40.645176 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d06542-6af9-4ea5-ad59-3331d37fde97 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:40.645835 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d06542-6af9-4ea5-ad59-3331d37fde97 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:16:40.646127 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d06542-6af9-4ea5-ad59-3331d37fde97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:40.647461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d06542-6af9-4ea5-ad59-3331d37fde97 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:40.654225 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 584/1166] 10.222.0.26 () {56 vars in 1115 bytes} [Fri Sep 9 20:16:40 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:42.790196 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da4a20-7a64-4732-b429-58fad2d2c85f None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:42.790708 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da4a20-7a64-4732-b429-58fad2d2c85f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:42.790708 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da4a20-7a64-4732-b429-58fad2d2c85f None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:42.792043 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 583/1167] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:16:42.797106 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462abdd-f9ea-4ea8-b01b-6ccc8d06d7f4 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:42.797569 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462abdd-f9ea-4ea8-b01b-6ccc8d06d7f4 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:42.797698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462abdd-f9ea-4ea8-b01b-6ccc8d06d7f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:42.816981 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-c462abdd-f9ea-4ea8-b01b-6ccc8d06d7f4 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:16:42.818227 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-c462abdd-f9ea-4ea8-b01b-6ccc8d06d7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:16:42.863600 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:42.884397 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 585/1168] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:16:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:16:43.150342 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da4a20-7a64-4732-b429-58fad2d2c85f None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:43.239118 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:43.249815 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da4a20-7a64-4732-b429-58fad2d2c85f None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:43.282910 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9be8b9-041c-4114-8250-ad025d1706c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:43.283501 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9be8b9-041c-4114-8250-ad025d1706c7 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:43.283775 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9be8b9-041c-4114-8250-ad025d1706c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:43.284040 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9be8b9-041c-4114-8250-ad025d1706c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:43.284690 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9be8b9-041c-4114-8250-ad025d1706c7 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:43.286083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9be8b9-041c-4114-8250-ad025d1706c7 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:43.300998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 584/1169] 10.222.0.26 () {60 vars in 1138 bytes} [Fri Sep 9 20:16:43 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:16:45.467407 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e74cb2-e666-4c35-91ae-ce4467cbb21f None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:45.467939 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e74cb2-e666-4c35-91ae-ce4467cbb21f None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:45.467939 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e74cb2-e666-4c35-91ae-ce4467cbb21f None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:45.469348 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 586/1170] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:16:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:16:45.475133 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb24a26-fa72-4912-aa1f-ff4eb19eab53 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:45.475549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb24a26-fa72-4912-aa1f-ff4eb19eab53 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:45.475680 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb24a26-fa72-4912-aa1f-ff4eb19eab53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:45.493395 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-9cb24a26-fa72-4912-aa1f-ff4eb19eab53 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:16:45.494575 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9cb24a26-fa72-4912-aa1f-ff4eb19eab53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:16:45.558982 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 585/1171] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:16:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:16:45.843593 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e74cb2-e666-4c35-91ae-ce4467cbb21f None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:45.946373 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e74cb2-e666-4c35-91ae-ce4467cbb21f None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:45.979946 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d392cf8b-f18a-48f6-867c-a1bb5d7b35f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:45.980566 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392cf8b-f18a-48f6-867c-a1bb5d7b35f9 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:45.980844 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392cf8b-f18a-48f6-867c-a1bb5d7b35f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:45.981117 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392cf8b-f18a-48f6-867c-a1bb5d7b35f9 None admin] PATH_INFO: `/v3/domains/52d56209d815483fa61bcd0e9d03e2ef` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:45.991611 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d392cf8b-f18a-48f6-867c-a1bb5d7b35f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=52d56209d815483fa61bcd0e9d03e2ef)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:45.993036 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d392cf8b-f18a-48f6-867c-a1bb5d7b35f9 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:45.994617 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 587/1172] 10.222.0.26 () {56 vars in 1183 bytes} [Fri Sep 9 20:16:45 2022] GET /identity/v3/domains/52d56209d815483fa61bcd0e9d03e2ef => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:46.000455 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb24a26-fa72-4912-aa1f-ff4eb19eab53 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:46.078396 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:46.098691 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb24a26-fa72-4912-aa1f-ff4eb19eab53 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:46.134932 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f9fc90-c8c5-457d-982e-ee14dcd8f9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:46.135555 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9fc90-c8c5-457d-982e-ee14dcd8f9c5 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:46.135830 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9fc90-c8c5-457d-982e-ee14dcd8f9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:46.136097 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9fc90-c8c5-457d-982e-ee14dcd8f9c5 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:46.136740 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f9fc90-c8c5-457d-982e-ee14dcd8f9c5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:46.138165 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f9fc90-c8c5-457d-982e-ee14dcd8f9c5 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:46.163719 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 586/1173] 10.222.0.26 () {60 vars in 1141 bytes} [Fri Sep 9 20:16:45 2022] POST /identity/v3/projects => generated 361 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:16:48.305963 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1329fa8-034a-4401-8147-c9c2566d7fd4 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:48.306553 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1329fa8-034a-4401-8147-c9c2566d7fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:48.306553 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1329fa8-034a-4401-8147-c9c2566d7fd4 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:48.307933 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 588/1174] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:16:48.313401 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55d0a4-d09e-4cb5-8c52-a232c6dbc330 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:48.313806 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55d0a4-d09e-4cb5-8c52-a232c6dbc330 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:48.313957 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55d0a4-d09e-4cb5-8c52-a232c6dbc330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:48.331650 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-7a55d0a4-d09e-4cb5-8c52-a232c6dbc330 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:16:48.333363 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7a55d0a4-d09e-4cb5-8c52-a232c6dbc330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:16:48.396892 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 587/1175] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:16:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:16:48.663533 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1329fa8-034a-4401-8147-c9c2566d7fd4 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:48.705263 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:48.763627 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1329fa8-034a-4401-8147-c9c2566d7fd4 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:48.795586 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0263d-4f04-431a-afff-42f434e1f5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:48.796201 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0263d-4f04-431a-afff-42f434e1f5e1 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:48.796477 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0263d-4f04-431a-afff-42f434e1f5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:48.796747 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0263d-4f04-431a-afff-42f434e1f5e1 None admin] PATH_INFO: `/v3/domains/52d56209d815483fa61bcd0e9d03e2ef` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:48.797840 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d0263d-4f04-431a-afff-42f434e1f5e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=52d56209d815483fa61bcd0e9d03e2ef)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:48.799253 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d0263d-4f04-431a-afff-42f434e1f5e1 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:48.800783 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 589/1176] 10.222.0.26 () {56 vars in 1183 bytes} [Fri Sep 9 20:16:48 2022] GET /identity/v3/domains/52d56209d815483fa61bcd0e9d03e2ef => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:48.806637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a55d0a4-d09e-4cb5-8c52-a232c6dbc330 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:48.865468 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:48.903276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a55d0a4-d09e-4cb5-8c52-a232c6dbc330 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:48.935714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a430b941-f32d-4e7b-89c3-bfb66ff0b818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:48.936262 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a430b941-f32d-4e7b-89c3-bfb66ff0b818 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:48.936538 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a430b941-f32d-4e7b-89c3-bfb66ff0b818 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:48.936811 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a430b941-f32d-4e7b-89c3-bfb66ff0b818 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:48.937473 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a430b941-f32d-4e7b-89c3-bfb66ff0b818 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:48.938879 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a430b941-f32d-4e7b-89c3-bfb66ff0b818 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:48.959043 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 588/1177] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Sep 9 20:16:48 2022] POST /identity/v3/users => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:16:51.102805 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc72a5f-306c-4563-a35f-daefe00c5ff3 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:51.103334 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc72a5f-306c-4563-a35f-daefe00c5ff3 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:51.103334 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc72a5f-306c-4563-a35f-daefe00c5ff3 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:51.104655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 590/1178] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:16:51.109860 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ef502-4c98-4a55-b5f6-9b3343337afb None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:51.110256 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ef502-4c98-4a55-b5f6-9b3343337afb None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:51.110404 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ef502-4c98-4a55-b5f6-9b3343337afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:51.130138 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-5a8ef502-4c98-4a55-b5f6-9b3343337afb None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:16:51.131294 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5a8ef502-4c98-4a55-b5f6-9b3343337afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:16:51.194259 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 589/1179] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:16:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:16:51.461089 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc72a5f-306c-4563-a35f-daefe00c5ff3 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:51.573715 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc72a5f-306c-4563-a35f-daefe00c5ff3 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:51.583246 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:51.610004 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927e9b85-1a7a-4841-a00e-c7cde6a5598d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:51.610626 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e9b85-1a7a-4841-a00e-c7cde6a5598d None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:51.610907 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e9b85-1a7a-4841-a00e-c7cde6a5598d None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:51.611182 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e9b85-1a7a-4841-a00e-c7cde6a5598d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:51.624023 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927e9b85-1a7a-4841-a00e-c7cde6a5598d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:16:51.629141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-927e9b85-1a7a-4841-a00e-c7cde6a5598d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:16:51.629141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:16:51.629141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:16:51.629141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:16:51.629141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:16:51.629141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:16:51.629141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:16:51.629141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:16:51.629141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:16:51.629141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:16:51.629141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:16:51.629141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:16:51.629141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:16:51.629141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:16:51.629141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:16:51.629141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:16:51.630345 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:16:51.630345 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:16:51.630345 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:16:51.630345 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:16:51.630345 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:16:51.630345 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:16:51.630345 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:16:51.630345 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:16:51.630345 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:16:51.630345 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:16:51.630345 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:16:51.630345 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:16:51.630345 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:16:51.630345 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:16:51.630345 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:16:51.630345 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:16:51.631281 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:16:51.631281 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:16:51.631281 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:16:51.631281 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:16:51.631281 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:16:51.631281 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:16:51.631281 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:16:51.631281 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:16:51.631281 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:16:51.631281 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:16:51.631281 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:16:51.631281 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:16:51.631281 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 591/1180] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:16:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:16:51.636282 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ef502-4c98-4a55-b5f6-9b3343337afb None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:51.689556 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:51.745095 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ef502-4c98-4a55-b5f6-9b3343337afb None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:51.781112 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603728b0-5884-4b7f-a8cb-24c207566ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:51.781915 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603728b0-5884-4b7f-a8cb-24c207566ecb None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:51.782427 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603728b0-5884-4b7f-a8cb-24c207566ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:51.782901 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603728b0-5884-4b7f-a8cb-24c207566ecb None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:51.783574 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603728b0-5884-4b7f-a8cb-24c207566ecb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:16:51.783913 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603728b0-5884-4b7f-a8cb-24c207566ecb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:51.785890 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603728b0-5884-4b7f-a8cb-24c207566ecb None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:51.793913 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 590/1181] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:16:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:51.800040 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927e9b85-1a7a-4841-a00e-c7cde6a5598d None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:51.910369 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927e9b85-1a7a-4841-a00e-c7cde6a5598d None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:51.945869 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa50825-b74b-42cc-a8bb-9a78cea4ba02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:51.946910 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa50825-b74b-42cc-a8bb-9a78cea4ba02 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:51.947398 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa50825-b74b-42cc-a8bb-9a78cea4ba02 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:51.947878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa50825-b74b-42cc-a8bb-9a78cea4ba02 None admin] PATH_INFO: `/v3/users/271b2e92161f44998690a76babfeb14c` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:51.961885 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa50825-b74b-42cc-a8bb-9a78cea4ba02 None admin] RBAC: Authorizing `identity:get_user(user_id=271b2e92161f44998690a76babfeb14c)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:51.963320 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa50825-b74b-42cc-a8bb-9a78cea4ba02 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:51.964934 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 592/1182] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:16:51 2022] GET /identity/v3/users/271b2e92161f44998690a76babfeb14c => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:51.971385 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603728b0-5884-4b7f-a8cb-24c207566ecb None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:52.075699 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603728b0-5884-4b7f-a8cb-24c207566ecb None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:52.087080 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:52.111417 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894b3f2-5187-44af-b7c5-41d2a8860e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:52.112439 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894b3f2-5187-44af-b7c5-41d2a8860e49 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:52.112943 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894b3f2-5187-44af-b7c5-41d2a8860e49 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:52.113487 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894b3f2-5187-44af-b7c5-41d2a8860e49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:52.183311 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0894b3f2-5187-44af-b7c5-41d2a8860e49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:52.184741 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0894b3f2-5187-44af-b7c5-41d2a8860e49 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:52.279044 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 591/1183] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:16:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:16:52.285129 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa50825-b74b-42cc-a8bb-9a78cea4ba02 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:52.311222 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:52.381573 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa50825-b74b-42cc-a8bb-9a78cea4ba02 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:52.414223 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1400356c-afe2-4436-a3db-dc470b6ea59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:52.414838 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400356c-afe2-4436-a3db-dc470b6ea59a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:52.415118 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400356c-afe2-4436-a3db-dc470b6ea59a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:52.415391 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400356c-afe2-4436-a3db-dc470b6ea59a None admin] PATH_INFO: `/v3/projects/0b4c8e6958604dbcb8f0be5d7aa46009` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:52.416483 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1400356c-afe2-4436-a3db-dc470b6ea59a None admin] RBAC: Authorizing `identity:get_project(project_id=0b4c8e6958604dbcb8f0be5d7aa46009)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:52.417938 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1400356c-afe2-4436-a3db-dc470b6ea59a None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:52.419541 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 593/1184] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:16:52 2022] GET /identity/v3/projects/0b4c8e6958604dbcb8f0be5d7aa46009 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:52.425578 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894b3f2-5187-44af-b7c5-41d2a8860e49 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:52.436741 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:52.521406 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894b3f2-5187-44af-b7c5-41d2a8860e49 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:52.553119 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a736c2-cbad-4d48-962d-645f3d299ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:52.553736 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a736c2-cbad-4d48-962d-645f3d299ba8 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:52.554019 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a736c2-cbad-4d48-962d-645f3d299ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:52.554292 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a736c2-cbad-4d48-962d-645f3d299ba8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:52.555001 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a736c2-cbad-4d48-962d-645f3d299ba8 None admin] RBAC: Adding query filter params (role.id=9121ba25a6d84756990f0c3a19e9fecb, scope.project.id=0b4c8e6958604dbcb8f0be5d7aa46009, user.id=271b2e92161f44998690a76babfeb14c) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:16:52.555304 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a736c2-cbad-4d48-962d-645f3d299ba8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:52.556703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a736c2-cbad-4d48-962d-645f3d299ba8 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:52.561838 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 592/1185] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:16:52 2022] GET /identity/v3/role_assignments?user.id=271b2e92161f44998690a76babfeb14c&scope.project.id=0b4c8e6958604dbcb8f0be5d7aa46009&role.id=9121ba25a6d84756990f0c3a19e9fecb => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:16:54.657815 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9663131-486c-4ccf-b9e3-483618cbbbb2 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:54.658342 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9663131-486c-4ccf-b9e3-483618cbbbb2 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:54.658342 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9663131-486c-4ccf-b9e3-483618cbbbb2 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:54.659683 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 594/1186] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:16:54.665431 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af0edc-636e-4775-8cbd-af45e24d3a74 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:54.665852 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af0edc-636e-4775-8cbd-af45e24d3a74 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:54.665980 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af0edc-636e-4775-8cbd-af45e24d3a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:54.684102 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a9af0edc-636e-4775-8cbd-af45e24d3a74 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:16:54.685271 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a9af0edc-636e-4775-8cbd-af45e24d3a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:16:54.750386 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 593/1187] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:16:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:16:55.023903 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9663131-486c-4ccf-b9e3-483618cbbbb2 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:55.114944 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:55.122597 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9663131-486c-4ccf-b9e3-483618cbbbb2 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:55.157939 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a56f14-d05d-4f6c-b0b7-6331f32beecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:55.158553 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a56f14-d05d-4f6c-b0b7-6331f32beecb None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:55.158829 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a56f14-d05d-4f6c-b0b7-6331f32beecb None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:55.159098 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a56f14-d05d-4f6c-b0b7-6331f32beecb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:55.169439 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a56f14-d05d-4f6c-b0b7-6331f32beecb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:16:55.174150 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-f6a56f14-d05d-4f6c-b0b7-6331f32beecb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:16:55.174150 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:16:55.174150 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:16:55.174150 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:16:55.174150 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:16:55.174150 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:16:55.174150 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:16:55.174150 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:16:55.174150 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:16:55.174150 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:16:55.174150 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:16:55.174150 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:16:55.174150 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:16:55.174150 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:16:55.174150 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:16:55.174150 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:16:55.175254 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:16:55.175254 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:16:55.175254 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:16:55.175254 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:16:55.175254 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:16:55.175254 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:16:55.175254 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:16:55.175254 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:16:55.175254 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:16:55.175254 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:16:55.175254 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:16:55.175254 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:16:55.175254 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:16:55.175254 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:16:55.175254 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:16:55.175254 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:16:55.176147 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:16:55.176147 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:16:55.176147 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:16:55.176147 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:16:55.176147 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:16:55.176147 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:16:55.176147 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:16:55.176147 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:16:55.176147 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:16:55.176147 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:16:55.176147 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:16:55.176147 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:16:55.176147 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 595/1188] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:16:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:16:55.181284 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af0edc-636e-4775-8cbd-af45e24d3a74 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:55.184309 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:55.276892 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af0edc-636e-4775-8cbd-af45e24d3a74 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:55.308590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57fef93-3ff8-4ffd-9f16-29a34f8b5537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:55.309181 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57fef93-3ff8-4ffd-9f16-29a34f8b5537 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:55.309608 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57fef93-3ff8-4ffd-9f16-29a34f8b5537 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:55.309881 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57fef93-3ff8-4ffd-9f16-29a34f8b5537 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:55.310531 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57fef93-3ff8-4ffd-9f16-29a34f8b5537 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:16:55.310828 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57fef93-3ff8-4ffd-9f16-29a34f8b5537 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:55.312173 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57fef93-3ff8-4ffd-9f16-29a34f8b5537 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:55.320517 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 594/1189] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:16:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:55.326866 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a56f14-d05d-4f6c-b0b7-6331f32beecb None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:55.423173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a56f14-d05d-4f6c-b0b7-6331f32beecb None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:55.455569 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c784bd-64ca-4bec-b918-c3e25a8d4fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:55.456177 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c784bd-64ca-4bec-b918-c3e25a8d4fd3 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:55.456454 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c784bd-64ca-4bec-b918-c3e25a8d4fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:55.456725 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c784bd-64ca-4bec-b918-c3e25a8d4fd3 None admin] PATH_INFO: `/v3/users/271b2e92161f44998690a76babfeb14c` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:55.458378 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c784bd-64ca-4bec-b918-c3e25a8d4fd3 None admin] RBAC: Authorizing `identity:get_user(user_id=271b2e92161f44998690a76babfeb14c)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:55.459785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c784bd-64ca-4bec-b918-c3e25a8d4fd3 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:55.461547 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 596/1190] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:16:55 2022] GET /identity/v3/users/271b2e92161f44998690a76babfeb14c => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:55.466969 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57fef93-3ff8-4ffd-9f16-29a34f8b5537 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:55.530879 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:55.564449 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57fef93-3ff8-4ffd-9f16-29a34f8b5537 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:55.597208 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9326f7ce-513a-4178-8b20-25a6757f9a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:55.597878 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326f7ce-513a-4178-8b20-25a6757f9a2d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:55.598162 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326f7ce-513a-4178-8b20-25a6757f9a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:55.598437 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326f7ce-513a-4178-8b20-25a6757f9a2d None admin] PATH_INFO: `/v3/projects/0b4c8e6958604dbcb8f0be5d7aa46009` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:55.599535 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9326f7ce-513a-4178-8b20-25a6757f9a2d None admin] RBAC: Authorizing `identity:get_project(project_id=0b4c8e6958604dbcb8f0be5d7aa46009)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:55.600943 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9326f7ce-513a-4178-8b20-25a6757f9a2d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:55.602562 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 595/1191] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:16:55 2022] GET /identity/v3/projects/0b4c8e6958604dbcb8f0be5d7aa46009 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:55.608297 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c784bd-64ca-4bec-b918-c3e25a8d4fd3 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:55.632996 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:55.705876 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c784bd-64ca-4bec-b918-c3e25a8d4fd3 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:55.737578 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55374601-f918-481f-bd1a-f22012aa5641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:55.738240 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55374601-f918-481f-bd1a-f22012aa5641 None admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:55.739020 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55374601-f918-481f-bd1a-f22012aa5641 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:55.739287 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55374601-f918-481f-bd1a-f22012aa5641 None admin] PATH_INFO: `/v3/projects/0b4c8e6958604dbcb8f0be5d7aa46009/users/271b2e92161f44998690a76babfeb14c/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:55.741119 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55374601-f918-481f-bd1a-f22012aa5641 None admin] RBAC: Authorizing `identity:create_grant(project_id=0b4c8e6958604dbcb8f0be5d7aa46009, user_id=271b2e92161f44998690a76babfeb14c, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:55.742557 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55374601-f918-481f-bd1a-f22012aa5641 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:55.755561 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 597/1192] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Sep 9 20:16:55 2022] PUT /identity/v3/projects/0b4c8e6958604dbcb8f0be5d7aa46009/users/271b2e92161f44998690a76babfeb14c/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:16:57.882590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987f577-f37b-4b58-a494-2cfe823469e6 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:57.883116 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987f577-f37b-4b58-a494-2cfe823469e6 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:57.883116 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987f577-f37b-4b58-a494-2cfe823469e6 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:57.884432 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 596/1193] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:16:57.889700 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352509e1-49f4-4a88-b0ad-e3229f9a5eec None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:57.890133 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352509e1-49f4-4a88-b0ad-e3229f9a5eec None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:57.890239 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352509e1-49f4-4a88-b0ad-e3229f9a5eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:57.907847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-352509e1-49f4-4a88-b0ad-e3229f9a5eec None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:16:57.909026 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-352509e1-49f4-4a88-b0ad-e3229f9a5eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:16:57.973917 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 598/1194] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:16:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:16:58.260443 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6987f577-f37b-4b58-a494-2cfe823469e6 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:58.374456 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6987f577-f37b-4b58-a494-2cfe823469e6 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:58.407175 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e0661-533c-4c28-8213-4a56840a5400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:58.407797 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e0661-533c-4c28-8213-4a56840a5400 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:58.408075 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e0661-533c-4c28-8213-4a56840a5400 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:58.408348 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e0661-533c-4c28-8213-4a56840a5400 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:58.410704 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:58.419668 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9e0661-533c-4c28-8213-4a56840a5400 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:16:58.424485 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-8d9e0661-533c-4c28-8213-4a56840a5400 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:16:58.424485 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:16:58.424485 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:16:58.424485 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:16:58.424485 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:16:58.424485 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:16:58.424485 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:16:58.424485 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:16:58.424485 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:16:58.424485 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:16:58.424485 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:16:58.424485 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:16:58.424485 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:16:58.424485 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:16:58.424485 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:16:58.424485 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:16:58.425558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:16:58.425558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:16:58.425558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:16:58.425558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:16:58.425558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:16:58.425558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:16:58.425558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:16:58.425558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:16:58.425558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:16:58.425558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:16:58.425558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:16:58.425558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:16:58.425558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:16:58.425558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:16:58.425558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:16:58.425558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:16:58.427008 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:16:58.427008 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:16:58.427008 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:16:58.427008 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:16:58.427008 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:16:58.427008 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:16:58.427008 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:16:58.427008 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:16:58.427008 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:16:58.427008 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:16:58.427008 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:16:58.427008 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:16:58.427008 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 597/1195] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:16:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:16:58.432342 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352509e1-49f4-4a88-b0ad-e3229f9a5eec None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:58.446140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:58.533720 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352509e1-49f4-4a88-b0ad-e3229f9a5eec None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:58.569008 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c30c7f-65b1-4b58-b2eb-65f481f54119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:58.569648 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c30c7f-65b1-4b58-b2eb-65f481f54119 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:58.569927 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c30c7f-65b1-4b58-b2eb-65f481f54119 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:58.570198 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c30c7f-65b1-4b58-b2eb-65f481f54119 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:58.570855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c30c7f-65b1-4b58-b2eb-65f481f54119 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:16:58.571151 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c30c7f-65b1-4b58-b2eb-65f481f54119 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:58.572503 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c30c7f-65b1-4b58-b2eb-65f481f54119 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:58.580213 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 599/1196] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:16:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:58.586736 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e0661-533c-4c28-8213-4a56840a5400 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:58.685273 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e0661-533c-4c28-8213-4a56840a5400 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:58.718278 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67a22d-0cb7-413b-ab90-6a44b9da5ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:58.718902 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67a22d-0cb7-413b-ab90-6a44b9da5ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:58.719186 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67a22d-0cb7-413b-ab90-6a44b9da5ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:58.719462 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67a22d-0cb7-413b-ab90-6a44b9da5ee4 None admin] PATH_INFO: `/v3/users/271b2e92161f44998690a76babfeb14c` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:58.720574 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e67a22d-0cb7-413b-ab90-6a44b9da5ee4 None admin] RBAC: Authorizing `identity:get_user(user_id=271b2e92161f44998690a76babfeb14c)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:58.722028 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e67a22d-0cb7-413b-ab90-6a44b9da5ee4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:58.724201 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 598/1197] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Sep 9 20:16:58 2022] GET /identity/v3/users/271b2e92161f44998690a76babfeb14c => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:58.729998 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c30c7f-65b1-4b58-b2eb-65f481f54119 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:58.808199 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:58.832472 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c30c7f-65b1-4b58-b2eb-65f481f54119 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:58.865150 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43643bf-df39-43d6-b59d-6c94bdfeb438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:58.865773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43643bf-df39-43d6-b59d-6c94bdfeb438 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:58.866051 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43643bf-df39-43d6-b59d-6c94bdfeb438 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:58.866324 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43643bf-df39-43d6-b59d-6c94bdfeb438 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:58.937176 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43643bf-df39-43d6-b59d-6c94bdfeb438 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:58.938757 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43643bf-df39-43d6-b59d-6c94bdfeb438 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:59.013998 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:59.038738 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 600/1198] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:16:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:16:59.044726 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67a22d-0cb7-413b-ab90-6a44b9da5ee4 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:59.103853 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:16:59.141808 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67a22d-0cb7-413b-ab90-6a44b9da5ee4 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:59.174777 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d0625-d64a-401c-a10e-938b9310ad06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:59.175391 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d0625-d64a-401c-a10e-938b9310ad06 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:59.175677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d0625-d64a-401c-a10e-938b9310ad06 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:59.175950 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d0625-d64a-401c-a10e-938b9310ad06 None admin] PATH_INFO: `/v3/projects/0b4c8e6958604dbcb8f0be5d7aa46009` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:59.177032 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551d0625-d64a-401c-a10e-938b9310ad06 None admin] RBAC: Authorizing `identity:get_project(project_id=0b4c8e6958604dbcb8f0be5d7aa46009)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:59.178606 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551d0625-d64a-401c-a10e-938b9310ad06 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:59.180636 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 599/1199] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Sep 9 20:16:59 2022] GET /identity/v3/projects/0b4c8e6958604dbcb8f0be5d7aa46009 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:16:59.186610 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43643bf-df39-43d6-b59d-6c94bdfeb438 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:16:59.284285 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43643bf-df39-43d6-b59d-6c94bdfeb438 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:16:59.317981 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d9c41-953f-49d9-93b2-a70cfc7e2586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:16:59.318570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d9c41-953f-49d9-93b2-a70cfc7e2586 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:16:59.318848 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d9c41-953f-49d9-93b2-a70cfc7e2586 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:16:59.319111 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d9c41-953f-49d9-93b2-a70cfc7e2586 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:16:59.319811 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77d9c41-953f-49d9-93b2-a70cfc7e2586 None admin] RBAC: Adding query filter params (role.id=9121ba25a6d84756990f0c3a19e9fecb, scope.project.id=0b4c8e6958604dbcb8f0be5d7aa46009, user.id=271b2e92161f44998690a76babfeb14c) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:16:59.320110 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77d9c41-953f-49d9-93b2-a70cfc7e2586 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:16:59.321528 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77d9c41-953f-49d9-93b2-a70cfc7e2586 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:16:59.327020 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 601/1200] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:16:59 2022] GET /identity/v3/role_assignments?user.id=271b2e92161f44998690a76babfeb14c&scope.project.id=0b4c8e6958604dbcb8f0be5d7aa46009&role.id=9121ba25a6d84756990f0c3a19e9fecb => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:17:09.170576 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=35 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:17:09.324493 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=81 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:17:55.068251 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc558e-af3d-400c-baaa-139c113951e8 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:17:55.068836 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc558e-af3d-400c-baaa-139c113951e8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:17:55.068933 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc558e-af3d-400c-baaa-139c113951e8 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:17:55.070890 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 600/1201] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:17:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:17:55.077666 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8424100-a994-4f4e-8ee6-0d83981b931d None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:17:55.078174 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8424100-a994-4f4e-8ee6-0d83981b931d None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:17:55.078256 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8424100-a994-4f4e-8ee6-0d83981b931d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:17:55.098425 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-e8424100-a994-4f4e-8ee6-0d83981b931d None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:17:55.099916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-e8424100-a994-4f4e-8ee6-0d83981b931d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:17:55.181675 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 602/1202] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:17:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:17:55.465482 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc558e-af3d-400c-baaa-139c113951e8 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:17:55.597387 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc558e-af3d-400c-baaa-139c113951e8 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:17:55.642771 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3213aa-5687-4b48-89cb-d4385d473907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:17:55.644117 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3213aa-5687-4b48-89cb-d4385d473907 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:17:55.644401 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3213aa-5687-4b48-89cb-d4385d473907 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:17:55.644674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3213aa-5687-4b48-89cb-d4385d473907 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:17:55.661158 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3213aa-5687-4b48-89cb-d4385d473907 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:17:55.662618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3213aa-5687-4b48-89cb-d4385d473907 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:17:55.681697 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-db3213aa-5687-4b48-89cb-d4385d473907 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:17:55.681697 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:17:55.681697 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:17:55.681697 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:17:55.681697 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:17:55.681697 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:17:55.681697 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:17:55.681697 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:17:55.681697 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:17:55.681697 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:17:55.681697 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:17:55.681697 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:17:55.681697 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:17:55.681697 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:17:55.681697 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:17:55.681697 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:17:55.682835 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:17:55.682835 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:17:55.682835 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:17:55.682835 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:17:55.682835 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:17:55.682835 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:17:55.682835 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:17:55.682835 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:17:55.682835 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:17:55.682835 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:17:55.682835 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:17:55.682835 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:17:55.682835 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:17:55.682835 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:17:55.682835 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:17:55.682835 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:17:55.682835 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:17:55.684489 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:17:55.684489 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:17:55.684489 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:17:55.684489 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 601/1203] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:17:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:17:55.689049 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8424100-a994-4f4e-8ee6-0d83981b931d None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:17:55.820941 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8424100-a994-4f4e-8ee6-0d83981b931d None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:17:55.859588 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:17:55.867378 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bdd54a-cb8b-45ee-8dc1-26591044ee7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:17:55.867990 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdd54a-cb8b-45ee-8dc1-26591044ee7d None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:17:55.868270 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdd54a-cb8b-45ee-8dc1-26591044ee7d None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:17:55.868838 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdd54a-cb8b-45ee-8dc1-26591044ee7d None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:17:55.870090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bdd54a-cb8b-45ee-8dc1-26591044ee7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:17:55.870090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bdd54a-cb8b-45ee-8dc1-26591044ee7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:17:55.871473 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bdd54a-cb8b-45ee-8dc1-26591044ee7d None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:17:55.888756 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 603/1204] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:17:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:17:55.890058 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3213aa-5687-4b48-89cb-d4385d473907 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:17:55.968922 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:17:56.035448 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3213aa-5687-4b48-89cb-d4385d473907 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:17:56.078802 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f0ce0-ac91-46d4-9306-a5bdd89a9c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:17:56.079455 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f0ce0-ac91-46d4-9306-a5bdd89a9c82 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:17:56.079689 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f0ce0-ac91-46d4-9306-a5bdd89a9c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:17:56.079933 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f0ce0-ac91-46d4-9306-a5bdd89a9c82 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:17:56.080659 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2f0ce0-ac91-46d4-9306-a5bdd89a9c82 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:17:56.082335 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2f0ce0-ac91-46d4-9306-a5bdd89a9c82 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:17:56.106301 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 602/1205] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Sep 9 20:17:55 2022] POST /identity/v3/users => generated 251 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:17:58.801789 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36df7e-6df3-4703-83a9-285d635d39f6 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:17:58.803056 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36df7e-6df3-4703-83a9-285d635d39f6 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:17:58.803056 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36df7e-6df3-4703-83a9-285d635d39f6 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:17:58.803908 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 604/1206] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:17:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:17:58.809747 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebac48a-3120-4316-bce5-3bc760b629e3 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:17:58.810242 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebac48a-3120-4316-bce5-3bc760b629e3 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:17:58.810348 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebac48a-3120-4316-bce5-3bc760b629e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:17:58.829725 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-bebac48a-3120-4316-bce5-3bc760b629e3 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:17:58.831691 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-bebac48a-3120-4316-bce5-3bc760b629e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:17:58.905892 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 603/1207] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:17:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:17:59.169815 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36df7e-6df3-4703-83a9-285d635d39f6 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:17:59.278419 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36df7e-6df3-4703-83a9-285d635d39f6 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:17:59.316371 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c9906-4a59-472d-bd9a-15bbb100ac0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:17:59.317568 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c9906-4a59-472d-bd9a-15bbb100ac0a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:17:59.317568 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c9906-4a59-472d-bd9a-15bbb100ac0a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:17:59.317781 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c9906-4a59-472d-bd9a-15bbb100ac0a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:17:59.328966 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4c9906-4a59-472d-bd9a-15bbb100ac0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:17:59.334551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-2c4c9906-4a59-472d-bd9a-15bbb100ac0a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:17:59.334551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:17:59.334551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:17:59.334551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:17:59.334551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:17:59.334551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:17:59.334551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:17:59.334551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:17:59.334551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:17:59.334551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:17:59.334551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:17:59.334551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:17:59.334551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:17:59.334551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:17:59.334551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:17:59.334551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:17:59.335680 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:17:59.335680 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:17:59.335680 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:17:59.335680 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:17:59.335680 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:17:59.335680 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:17:59.335680 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:17:59.335680 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:17:59.335680 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:17:59.335680 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:17:59.335680 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:17:59.335680 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:17:59.335680 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:17:59.335680 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:17:59.335680 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:17:59.335680 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:17:59.338275 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:17:59.338275 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:17:59.338275 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:17:59.338275 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:17:59.338275 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:17:59.338275 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:17:59.338275 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:17:59.338275 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:17:59.338275 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:17:59.338275 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:17:59.338275 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:17:59.338275 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:17:59.338275 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 605/1208] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Sep 9 20:17:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:17:59.341819 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebac48a-3120-4316-bce5-3bc760b629e3 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:17:59.385825 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:17:59.452702 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebac48a-3120-4316-bce5-3bc760b629e3 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:17:59.488688 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193ced3-6285-4c49-933a-e6274b678454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:17:59.489332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193ced3-6285-4c49-933a-e6274b678454 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:17:59.489618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193ced3-6285-4c49-933a-e6274b678454 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:17:59.489889 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193ced3-6285-4c49-933a-e6274b678454 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:17:59.490562 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193ced3-6285-4c49-933a-e6274b678454 None admin] RBAC: Adding query filter params (name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:17:59.490894 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193ced3-6285-4c49-933a-e6274b678454 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:17:59.492223 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193ced3-6285-4c49-933a-e6274b678454 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:17:59.500409 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 604/1209] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Sep 9 20:17:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:17:59.506607 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c9906-4a59-472d-bd9a-15bbb100ac0a None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:17:59.560923 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:17:59.620814 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c9906-4a59-472d-bd9a-15bbb100ac0a None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:17:59.670781 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c9fb4-0444-43f9-ae8e-e7f3bdfafbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:17:59.671462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c9fb4-0444-43f9-ae8e-e7f3bdfafbcc None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:17:59.671743 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c9fb4-0444-43f9-ae8e-e7f3bdfafbcc None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:17:59.672017 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c9fb4-0444-43f9-ae8e-e7f3bdfafbcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:17:59.684417 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c9fb4-0444-43f9-ae8e-e7f3bdfafbcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:17:59.686078 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c9fb4-0444-43f9-ae8e-e7f3bdfafbcc None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:17:59.703088 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-ff4c9fb4-0444-43f9-ae8e-e7f3bdfafbcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:17:59.703088 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:17:59.703088 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:17:59.703088 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:17:59.703088 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:17:59.703088 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:17:59.703088 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:17:59.703088 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:17:59.703088 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:17:59.703088 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:17:59.703088 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:17:59.703088 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:17:59.703088 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:17:59.703088 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:17:59.703088 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:17:59.703088 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:17:59.704259 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:17:59.704259 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:17:59.704259 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:17:59.704259 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:17:59.704259 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:17:59.704259 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:17:59.704259 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:17:59.704259 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:17:59.704259 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:17:59.704259 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:17:59.704259 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:17:59.704259 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:17:59.704259 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:17:59.704259 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:17:59.704259 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:17:59.704259 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:17:59.704259 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:17:59.705169 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:17:59.705169 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:17:59.705169 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:17:59.705169 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 606/1210] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:17:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:17:59.710697 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193ced3-6285-4c49-933a-e6274b678454 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:17:59.844016 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:17:59.847113 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193ced3-6285-4c49-933a-e6274b678454 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:17:59.888658 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63326f1-e689-4d5f-8555-a9063e6b0b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:17:59.890118 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63326f1-e689-4d5f-8555-a9063e6b0b4b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:17:59.890118 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63326f1-e689-4d5f-8555-a9063e6b0b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:17:59.890118 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63326f1-e689-4d5f-8555-a9063e6b0b4b None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:17:59.890658 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63326f1-e689-4d5f-8555-a9063e6b0b4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:17:59.890855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63326f1-e689-4d5f-8555-a9063e6b0b4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:17:59.892200 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63326f1-e689-4d5f-8555-a9063e6b0b4b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:17:59.905593 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 605/1211] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:17:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:17:59.911499 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c9fb4-0444-43f9-ae8e-e7f3bdfafbcc None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:00.026468 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c9fb4-0444-43f9-ae8e-e7f3bdfafbcc None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:00.028015 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:00.074642 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6186649-7845-499a-a8e9-7a8f8861ddb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:00.075387 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6186649-7845-499a-a8e9-7a8f8861ddb0 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:00.075656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6186649-7845-499a-a8e9-7a8f8861ddb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:00.076109 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6186649-7845-499a-a8e9-7a8f8861ddb0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:00.089525 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6186649-7845-499a-a8e9-7a8f8861ddb0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:00.091007 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6186649-7845-499a-a8e9-7a8f8861ddb0 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:00.104736 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-e6186649-7845-499a-a8e9-7a8f8861ddb0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 20:18:00.104736 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:18:00.104736 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:18:00.104736 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:18:00.104736 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:18:00.104736 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:18:00.104736 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:18:00.104736 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:18:00.104736 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:18:00.104736 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:18:00.104736 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:18:00.104736 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:18:00.104736 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:18:00.104736 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:18:00.104736 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:18:00.104736 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:18:00.105872 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:18:00.105872 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:18:00.105872 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:18:00.105872 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:18:00.105872 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:18:00.105872 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:18:00.105872 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:18:00.105872 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:18:00.105872 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:18:00.105872 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:18:00.105872 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:18:00.105872 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:18:00.105872 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:18:00.105872 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:18:00.105872 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:18:00.105872 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:18:00.105872 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:18:00.105872 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:18:00.106895 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:18:00.106895 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:18:00.106895 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:18:00.106895 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:18:00.106895 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:18:00.106895 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:18:00.106895 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:18:00.106895 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:18:00.106895 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 20:18:00.106895 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:18:00.106895 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 607/1212] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:17:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:18:00.113077 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63326f1-e689-4d5f-8555-a9063e6b0b4b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:00.235075 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63326f1-e689-4d5f-8555-a9063e6b0b4b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:00.276242 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5312f5-958d-43ae-aafd-fe000b2cb9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:00.276973 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5312f5-958d-43ae-aafd-fe000b2cb9d7 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:00.277197 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5312f5-958d-43ae-aafd-fe000b2cb9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:00.277548 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5312f5-958d-43ae-aafd-fe000b2cb9d7 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:00.278347 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5312f5-958d-43ae-aafd-fe000b2cb9d7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:00.278642 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5312f5-958d-43ae-aafd-fe000b2cb9d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:00.280144 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5312f5-958d-43ae-aafd-fe000b2cb9d7 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:00.303505 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 606/1213] 10.222.0.26 () {56 vars in 1143 bytes} [Fri Sep 9 20:18:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:18:00.305238 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6186649-7845-499a-a8e9-7a8f8861ddb0 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:00.435829 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6186649-7845-499a-a8e9-7a8f8861ddb0 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:00.473055 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef122c07-ce38-4761-9666-f0d68d5e664c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:00.473683 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef122c07-ce38-4761-9666-f0d68d5e664c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:00.473964 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef122c07-ce38-4761-9666-f0d68d5e664c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:00.474234 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef122c07-ce38-4761-9666-f0d68d5e664c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:00.507952 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:00.547999 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef122c07-ce38-4761-9666-f0d68d5e664c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:00.550111 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef122c07-ce38-4761-9666-f0d68d5e664c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:00.686135 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 608/1214] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:18:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:18:00.692443 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5312f5-958d-43ae-aafd-fe000b2cb9d7 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:00.718752 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:00.814953 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5312f5-958d-43ae-aafd-fe000b2cb9d7 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:00.857597 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb98e6-6099-49e1-85ca-f99a64488090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:00.859068 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb98e6-6099-49e1-85ca-f99a64488090 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:00.859337 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb98e6-6099-49e1-85ca-f99a64488090 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:00.859606 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb98e6-6099-49e1-85ca-f99a64488090 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:00.871068 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bb98e6-6099-49e1-85ca-f99a64488090 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:00.873008 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bb98e6-6099-49e1-85ca-f99a64488090 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:00.888518 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-31bb98e6-6099-49e1-85ca-f99a64488090 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:18:00.888518 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:18:00.888518 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:18:00.888518 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:18:00.888518 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:18:00.888518 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:18:00.888518 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:18:00.888518 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:18:00.888518 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:18:00.888518 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:18:00.888518 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:18:00.888518 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:18:00.888518 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:18:00.888518 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:18:00.888518 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:18:00.888518 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:18:00.889705 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:18:00.889705 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:18:00.889705 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:18:00.889705 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:18:00.889705 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:18:00.889705 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:18:00.889705 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:18:00.889705 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:18:00.889705 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:18:00.889705 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:18:00.889705 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:18:00.889705 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:18:00.889705 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:18:00.889705 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:18:00.889705 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:18:00.889705 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:18:00.889705 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:18:00.890692 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:18:00.890692 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:18:00.890692 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:18:00.893899 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 607/1215] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:18:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:18:00.895806 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef122c07-ce38-4761-9666-f0d68d5e664c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:00.979363 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:01.028008 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef122c07-ce38-4761-9666-f0d68d5e664c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:01.072775 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd4401-30b6-4afa-8320-b7f29f2f03d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:01.073369 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd4401-30b6-4afa-8320-b7f29f2f03d2 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:01.073665 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd4401-30b6-4afa-8320-b7f29f2f03d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:01.074027 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd4401-30b6-4afa-8320-b7f29f2f03d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:01.074809 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cd4401-30b6-4afa-8320-b7f29f2f03d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:01.075230 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cd4401-30b6-4afa-8320-b7f29f2f03d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:01.076643 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cd4401-30b6-4afa-8320-b7f29f2f03d2 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:01.091699 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 609/1216] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:18:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:18:01.099190 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb98e6-6099-49e1-85ca-f99a64488090 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:01.186701 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:01.230490 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb98e6-6099-49e1-85ca-f99a64488090 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:01.279851 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154faa4f-d8ac-4717-8bd8-dff0239a60eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:01.280580 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154faa4f-d8ac-4717-8bd8-dff0239a60eb None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:01.280863 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154faa4f-d8ac-4717-8bd8-dff0239a60eb None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:01.281413 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154faa4f-d8ac-4717-8bd8-dff0239a60eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:01.289687 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154faa4f-d8ac-4717-8bd8-dff0239a60eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:01.291587 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154faa4f-d8ac-4717-8bd8-dff0239a60eb None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:01.305536 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-154faa4f-d8ac-4717-8bd8-dff0239a60eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:18:01.305536 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:18:01.305536 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:18:01.305536 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:18:01.305536 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:18:01.305536 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:18:01.305536 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:18:01.305536 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:18:01.305536 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:18:01.305536 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:18:01.305536 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:18:01.305536 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:18:01.305536 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:18:01.305536 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:18:01.305536 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:18:01.306688 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:18:01.306688 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:18:01.306688 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:18:01.306688 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:18:01.306688 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:18:01.306688 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:18:01.306688 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:18:01.306688 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:18:01.306688 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:18:01.306688 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:18:01.306688 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:18:01.306688 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:18:01.306688 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:18:01.306688 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:18:01.306688 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:18:01.306688 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:18:01.306688 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:18:01.307837 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:18:01.307837 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:18:01.307837 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:18:01.307837 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:18:01.307837 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:18:01.307837 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:18:01.307837 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:18:01.307837 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:18:01.307837 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 608/1217] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:18:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:18:01.313968 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd4401-30b6-4afa-8320-b7f29f2f03d2 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:01.435281 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd4401-30b6-4afa-8320-b7f29f2f03d2 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:01.461047 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:01.474812 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e8e37-5b70-4471-a393-284b93f21aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:01.475514 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e8e37-5b70-4471-a393-284b93f21aff None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:01.475863 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e8e37-5b70-4471-a393-284b93f21aff None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:01.476171 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e8e37-5b70-4471-a393-284b93f21aff None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:01.477090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7e8e37-5b70-4471-a393-284b93f21aff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:01.477438 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7e8e37-5b70-4471-a393-284b93f21aff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:01.479324 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7e8e37-5b70-4471-a393-284b93f21aff None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:01.492604 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 610/1218] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:18:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:18:01.498630 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154faa4f-d8ac-4717-8bd8-dff0239a60eb None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:01.605054 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154faa4f-d8ac-4717-8bd8-dff0239a60eb None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:01.642505 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:01.649926 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437005ff-c2f4-4116-8ff8-2ba047671c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:01.651018 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437005ff-c2f4-4116-8ff8-2ba047671c4e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:01.651355 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437005ff-c2f4-4116-8ff8-2ba047671c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:01.651629 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437005ff-c2f4-4116-8ff8-2ba047671c4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:01.652345 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437005ff-c2f4-4116-8ff8-2ba047671c4e None admin] RBAC: Adding query filter params (role.id=8ec32f702e0c447993ff294cbac536b1, scope.project.id=933820e2adbc473bbd57a4d59be75d72, user.id=26d24cf0a21d491082b0c2f11e749446) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:01.652665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437005ff-c2f4-4116-8ff8-2ba047671c4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:01.654097 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437005ff-c2f4-4116-8ff8-2ba047671c4e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:01.660903 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 609/1219] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:18:01 2022] GET /identity/v3/role_assignments?user.id=26d24cf0a21d491082b0c2f11e749446&scope.project.id=933820e2adbc473bbd57a4d59be75d72&role.id=8ec32f702e0c447993ff294cbac536b1 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:18:03.910059 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a965b-da94-41f3-a70f-34610f1d1065 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:03.910607 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a965b-da94-41f3-a70f-34610f1d1065 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:03.911061 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a965b-da94-41f3-a70f-34610f1d1065 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:03.912642 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 611/1220] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:18:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:18:03.918169 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd67889-0dee-4609-910b-bb204df0a6d7 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:03.918593 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd67889-0dee-4609-910b-bb204df0a6d7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:03.918734 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd67889-0dee-4609-910b-bb204df0a6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:03.937596 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-5fd67889-0dee-4609-910b-bb204df0a6d7 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:18:03.938714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5fd67889-0dee-4609-910b-bb204df0a6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:18:04.014335 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 610/1221] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:18:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:18:04.319940 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a965b-da94-41f3-a70f-34610f1d1065 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:04.434150 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a965b-da94-41f3-a70f-34610f1d1065 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:04.471888 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63384e-3510-47b2-b5c2-5df4aa8d40e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:04.472516 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63384e-3510-47b2-b5c2-5df4aa8d40e0 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:04.472793 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63384e-3510-47b2-b5c2-5df4aa8d40e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:04.473072 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63384e-3510-47b2-b5c2-5df4aa8d40e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:04.484290 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d63384e-3510-47b2-b5c2-5df4aa8d40e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:18:04.489733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-0d63384e-3510-47b2-b5c2-5df4aa8d40e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:18:04.489733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:18:04.489733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:18:04.489733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:18:04.489733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:18:04.489733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:18:04.489733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:18:04.489733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:18:04.489733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:18:04.489733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:18:04.489733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:18:04.489733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:18:04.489733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:18:04.489733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:18:04.489733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:18:04.489733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:18:04.490887 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:18:04.490887 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:18:04.490887 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:18:04.490887 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:18:04.490887 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:18:04.490887 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:18:04.490887 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:18:04.490887 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:18:04.490887 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:18:04.490887 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:18:04.490887 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:18:04.490887 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:18:04.490887 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:18:04.490887 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:18:04.490887 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:18:04.490887 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:18:04.491853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:18:04.491853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:18:04.491853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:18:04.491853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:18:04.491853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:18:04.491853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:18:04.491853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:18:04.491853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:18:04.491853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:18:04.491853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:18:04.491853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:18:04.491853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:18:04.491853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 612/1222] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Sep 9 20:18:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:18:04.497753 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd67889-0dee-4609-910b-bb204df0a6d7 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:04.618452 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd67889-0dee-4609-910b-bb204df0a6d7 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:04.643770 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:04.655567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d70c0c9-b130-4c1c-b929-a179b5e258d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:04.656183 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70c0c9-b130-4c1c-b929-a179b5e258d1 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:04.656468 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70c0c9-b130-4c1c-b929-a179b5e258d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:04.656742 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70c0c9-b130-4c1c-b929-a179b5e258d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:04.657440 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d70c0c9-b130-4c1c-b929-a179b5e258d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:04.657777 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d70c0c9-b130-4c1c-b929-a179b5e258d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:04.659110 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d70c0c9-b130-4c1c-b929-a179b5e258d1 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:04.668930 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 611/1223] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Sep 9 20:18:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:18:04.675021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63384e-3510-47b2-b5c2-5df4aa8d40e0 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:04.723019 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:04.788942 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63384e-3510-47b2-b5c2-5df4aa8d40e0 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:04.826627 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0837d9ab-28bf-416d-ba2b-e25fc34fe038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:04.827262 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0837d9ab-28bf-416d-ba2b-e25fc34fe038 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:04.827540 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0837d9ab-28bf-416d-ba2b-e25fc34fe038 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:04.827815 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0837d9ab-28bf-416d-ba2b-e25fc34fe038 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:04.838879 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0837d9ab-28bf-416d-ba2b-e25fc34fe038 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:04.840336 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0837d9ab-28bf-416d-ba2b-e25fc34fe038 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:04.855541 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-0837d9ab-28bf-416d-ba2b-e25fc34fe038 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:18:04.855541 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:18:04.855541 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:18:04.855541 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:18:04.855541 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:18:04.855541 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:18:04.855541 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:18:04.855541 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:18:04.855541 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:18:04.855541 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:18:04.855541 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:18:04.855541 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:18:04.855541 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:18:04.855541 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:18:04.855541 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:18:04.855541 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:18:04.856788 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:18:04.856788 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:18:04.856788 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:18:04.856788 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:18:04.856788 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:18:04.856788 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:18:04.856788 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:18:04.856788 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:18:04.856788 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:18:04.856788 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:18:04.856788 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:18:04.856788 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:18:04.856788 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:18:04.856788 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:18:04.856788 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:18:04.856788 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:18:04.856788 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:18:04.857923 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:18:04.857923 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:18:04.857923 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:18:04.861490 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 613/1224] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:18:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:18:04.863047 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d70c0c9-b130-4c1c-b929-a179b5e258d1 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:04.977354 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d70c0c9-b130-4c1c-b929-a179b5e258d1 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:05.016887 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda54315-11a6-4a0d-8b04-c0c5f6bc89ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:05.017527 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda54315-11a6-4a0d-8b04-c0c5f6bc89ea None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:05.017832 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda54315-11a6-4a0d-8b04-c0c5f6bc89ea None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:05.018625 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda54315-11a6-4a0d-8b04-c0c5f6bc89ea None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:05.019262 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda54315-11a6-4a0d-8b04-c0c5f6bc89ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:05.019601 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda54315-11a6-4a0d-8b04-c0c5f6bc89ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:05.020956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda54315-11a6-4a0d-8b04-c0c5f6bc89ea None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:05.032981 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 612/1225] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:18:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:18:05.038904 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0837d9ab-28bf-416d-ba2b-e25fc34fe038 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:05.148303 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:05.158175 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0837d9ab-28bf-416d-ba2b-e25fc34fe038 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:05.196890 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c61d5-9f49-4f66-826f-6d4d17331d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:05.197758 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c61d5-9f49-4f66-826f-6d4d17331d62 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:05.198042 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c61d5-9f49-4f66-826f-6d4d17331d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:05.198335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c61d5-9f49-4f66-826f-6d4d17331d62 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:05.207576 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8c61d5-9f49-4f66-826f-6d4d17331d62 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:05.209024 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8c61d5-9f49-4f66-826f-6d4d17331d62 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:05.222966 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-7a8c61d5-9f49-4f66-826f-6d4d17331d62 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 20:18:05.222966 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:18:05.222966 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:18:05.222966 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:18:05.222966 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:18:05.222966 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:18:05.222966 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:18:05.222966 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:18:05.222966 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:18:05.222966 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:18:05.222966 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:18:05.222966 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:18:05.222966 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:18:05.222966 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:18:05.222966 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:18:05.222966 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:18:05.224190 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:18:05.224190 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:18:05.224190 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:18:05.224190 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:18:05.224190 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:18:05.224190 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:18:05.224190 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:18:05.224190 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:18:05.224190 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:18:05.224190 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:18:05.224190 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:18:05.224190 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:18:05.224190 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:18:05.224190 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:18:05.224190 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:18:05.224190 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:18:05.224190 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:18:05.224190 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:18:05.225394 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:18:05.225394 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:18:05.225394 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:18:05.225394 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:18:05.225394 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:18:05.225394 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:18:05.225394 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:18:05.225394 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:18:05.225394 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 20:18:05.225394 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:18:05.225394 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 614/1226] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:18:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:18:05.230694 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda54315-11a6-4a0d-8b04-c0c5f6bc89ea None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:05.283896 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:05.344160 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda54315-11a6-4a0d-8b04-c0c5f6bc89ea None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:05.383418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6adde74-82c4-48f4-834c-610d68602263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:05.383984 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6adde74-82c4-48f4-834c-610d68602263 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:05.384330 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6adde74-82c4-48f4-834c-610d68602263 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:05.384686 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6adde74-82c4-48f4-834c-610d68602263 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:05.385436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6adde74-82c4-48f4-834c-610d68602263 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:05.385767 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6adde74-82c4-48f4-834c-610d68602263 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:05.387246 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6adde74-82c4-48f4-834c-610d68602263 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:05.406357 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 613/1227] 10.222.0.26 () {56 vars in 1143 bytes} [Fri Sep 9 20:18:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:18:05.412218 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c61d5-9f49-4f66-826f-6d4d17331d62 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:05.520671 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c61d5-9f49-4f66-826f-6d4d17331d62 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:05.555839 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036ca5f-d8b0-4dfb-a5a0-f51b23d6ac18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:05.556478 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036ca5f-d8b0-4dfb-a5a0-f51b23d6ac18 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:05.556776 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036ca5f-d8b0-4dfb-a5a0-f51b23d6ac18 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:05.557069 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036ca5f-d8b0-4dfb-a5a0-f51b23d6ac18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:05.568247 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d036ca5f-d8b0-4dfb-a5a0-f51b23d6ac18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:05.569826 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d036ca5f-d8b0-4dfb-a5a0-f51b23d6ac18 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:05.584860 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-d036ca5f-d8b0-4dfb-a5a0-f51b23d6ac18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:18:05.584860 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:18:05.584860 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:18:05.584860 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:18:05.584860 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:18:05.584860 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:18:05.584860 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:18:05.584860 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:18:05.584860 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:18:05.584860 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:18:05.584860 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:18:05.584860 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:18:05.584860 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:18:05.584860 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:18:05.584860 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:18:05.584860 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:18:05.586048 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:18:05.586048 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:18:05.586048 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:18:05.586048 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:18:05.586048 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:18:05.586048 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:18:05.586048 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:18:05.586048 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:18:05.586048 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:18:05.586048 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:18:05.586048 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:18:05.586048 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:18:05.586048 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:18:05.586048 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:18:05.586048 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:18:05.586048 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:18:05.586048 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:18:05.586983 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:18:05.586983 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:18:05.586983 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:18:05.586983 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 615/1228] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:18:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:18:05.591855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6adde74-82c4-48f4-834c-610d68602263 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:05.697429 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6adde74-82c4-48f4-834c-610d68602263 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:05.700234 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:05.737444 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872c505-b53f-4043-8ca2-b68617b178e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:05.738061 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872c505-b53f-4043-8ca2-b68617b178e3 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:05.738345 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872c505-b53f-4043-8ca2-b68617b178e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:05.738649 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872c505-b53f-4043-8ca2-b68617b178e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:05.739266 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872c505-b53f-4043-8ca2-b68617b178e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:05.739564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872c505-b53f-4043-8ca2-b68617b178e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:05.740917 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872c505-b53f-4043-8ca2-b68617b178e3 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:05.754128 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 614/1229] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:18:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:18:05.761110 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036ca5f-d8b0-4dfb-a5a0-f51b23d6ac18 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:05.771737 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:05.871568 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036ca5f-d8b0-4dfb-a5a0-f51b23d6ac18 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:05.906971 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b639c254-999e-4d86-8ca0-bc4a1aa9fc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:05.907622 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639c254-999e-4d86-8ca0-bc4a1aa9fc7b None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:05.907950 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639c254-999e-4d86-8ca0-bc4a1aa9fc7b None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:05.908220 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639c254-999e-4d86-8ca0-bc4a1aa9fc7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:05.915745 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639c254-999e-4d86-8ca0-bc4a1aa9fc7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:05.917181 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639c254-999e-4d86-8ca0-bc4a1aa9fc7b None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:05.929441 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-b639c254-999e-4d86-8ca0-bc4a1aa9fc7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:18:05.929441 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:18:05.929441 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:18:05.929441 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:18:05.929441 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:18:05.929441 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:18:05.929441 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:18:05.929441 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:18:05.929441 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:18:05.929441 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:18:05.929441 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:18:05.929441 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:18:05.929441 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:18:05.929441 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:18:05.929441 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:18:05.930570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:18:05.930570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:18:05.930570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:18:05.930570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:18:05.930570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:18:05.930570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:18:05.930570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:18:05.930570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:18:05.930570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:18:05.930570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:18:05.930570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:18:05.930570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:18:05.930570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:18:05.930570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:18:05.930570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:18:05.930570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:18:05.930570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:18:05.931974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:18:05.931974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:18:05.931974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:18:05.931974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:18:05.931974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:18:05.931974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:18:05.931974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:18:05.931974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:18:05.931974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 616/1230] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:18:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:18:05.936482 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872c505-b53f-4043-8ca2-b68617b178e3 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:06.049214 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872c505-b53f-4043-8ca2-b68617b178e3 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:06.091351 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5d015d-0ac5-46c2-95f5-9fcfd9639b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:06.091940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d015d-0ac5-46c2-95f5-9fcfd9639b40 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:06.092258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d015d-0ac5-46c2-95f5-9fcfd9639b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:06.092555 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d015d-0ac5-46c2-95f5-9fcfd9639b40 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:06.093363 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5d015d-0ac5-46c2-95f5-9fcfd9639b40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:06.093721 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5d015d-0ac5-46c2-95f5-9fcfd9639b40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:06.095202 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5d015d-0ac5-46c2-95f5-9fcfd9639b40 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:06.107741 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 615/1231] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:18:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:18:06.114304 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b639c254-999e-4d86-8ca0-bc4a1aa9fc7b None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:06.185599 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:06.226267 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b639c254-999e-4d86-8ca0-bc4a1aa9fc7b None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:06.263010 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f177c42a-6405-43ac-8d86-30230fff7ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:06.263736 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177c42a-6405-43ac-8d86-30230fff7ebb None admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:06.264080 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177c42a-6405-43ac-8d86-30230fff7ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:06.264423 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177c42a-6405-43ac-8d86-30230fff7ebb None admin] PATH_INFO: `/v3/projects/933820e2adbc473bbd57a4d59be75d72/users/26d24cf0a21d491082b0c2f11e749446/roles/8ec32f702e0c447993ff294cbac536b1` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:06.280333 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f177c42a-6405-43ac-8d86-30230fff7ebb None admin] RBAC: Authorizing `identity:create_grant(project_id=933820e2adbc473bbd57a4d59be75d72, user_id=26d24cf0a21d491082b0c2f11e749446, role_id=8ec32f702e0c447993ff294cbac536b1)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:06.281925 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f177c42a-6405-43ac-8d86-30230fff7ebb None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:06.295937 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 617/1232] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Sep 9 20:18:06 2022] PUT /identity/v3/projects/933820e2adbc473bbd57a4d59be75d72/users/26d24cf0a21d491082b0c2f11e749446/roles/8ec32f702e0c447993ff294cbac536b1 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:18:08.600793 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b1a9e-676d-46d6-8379-f7345098cac0 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:08.601427 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b1a9e-676d-46d6-8379-f7345098cac0 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:08.601427 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b1a9e-676d-46d6-8379-f7345098cac0 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:08.602854 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 616/1233] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:18:08.608521 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4f4ff-377e-44b1-a228-24509a99d178 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:08.608949 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4f4ff-377e-44b1-a228-24509a99d178 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:08.609049 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4f4ff-377e-44b1-a228-24509a99d178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:08.627726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-ddd4f4ff-377e-44b1-a228-24509a99d178 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:18:08.628899 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-ddd4f4ff-377e-44b1-a228-24509a99d178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:18:08.705936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 618/1234] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:18:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:18:09.007731 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b1a9e-676d-46d6-8379-f7345098cac0 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:09.041204 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:09.129033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b1a9e-676d-46d6-8379-f7345098cac0 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:09.168051 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56fba76-22c5-4069-a935-009b6f0c2564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:09.168696 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56fba76-22c5-4069-a935-009b6f0c2564 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:09.168975 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56fba76-22c5-4069-a935-009b6f0c2564 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:09.169278 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56fba76-22c5-4069-a935-009b6f0c2564 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:09.180693 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56fba76-22c5-4069-a935-009b6f0c2564 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:18:09.185794 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-e56fba76-22c5-4069-a935-009b6f0c2564 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:18:09.185794 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:18:09.185794 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:18:09.185794 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:18:09.185794 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:18:09.185794 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:18:09.185794 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:18:09.185794 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:18:09.185794 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:18:09.185794 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:18:09.185794 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:18:09.185794 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:18:09.185794 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:18:09.185794 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:18:09.185794 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:18:09.185794 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:18:09.189678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:18:09.189678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:18:09.189678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:18:09.189678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:18:09.189678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:18:09.189678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:18:09.189678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:18:09.189678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:18:09.189678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:18:09.189678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:18:09.189678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:18:09.189678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:18:09.189678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:18:09.189678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:18:09.189678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:18:09.189678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:18:09.191191 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:18:09.191191 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:18:09.191191 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:18:09.191191 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:18:09.191191 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:18:09.191191 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:18:09.191191 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:18:09.191191 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:18:09.191191 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:18:09.191191 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:18:09.191191 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:18:09.191191 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:18:09.191191 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 617/1235] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Sep 9 20:18:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:18:09.193915 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4f4ff-377e-44b1-a228-24509a99d178 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:09.240112 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:09.294569 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4f4ff-377e-44b1-a228-24509a99d178 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:09.331945 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15395334-0fd1-4ce1-8f50-2f6f547a9071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:09.332558 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15395334-0fd1-4ce1-8f50-2f6f547a9071 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:09.332837 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15395334-0fd1-4ce1-8f50-2f6f547a9071 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:09.333114 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15395334-0fd1-4ce1-8f50-2f6f547a9071 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:09.333893 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15395334-0fd1-4ce1-8f50-2f6f547a9071 None admin] RBAC: Adding query filter params (name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:09.334238 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15395334-0fd1-4ce1-8f50-2f6f547a9071 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:09.335809 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15395334-0fd1-4ce1-8f50-2f6f547a9071 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:09.344434 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 619/1236] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Sep 9 20:18:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:18:09.350879 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56fba76-22c5-4069-a935-009b6f0c2564 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:09.438070 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:09.455128 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56fba76-22c5-4069-a935-009b6f0c2564 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:09.493627 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa167ed-f158-4b94-85d6-3151436c8dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:09.494445 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa167ed-f158-4b94-85d6-3151436c8dea None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:09.494716 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa167ed-f158-4b94-85d6-3151436c8dea None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:09.495094 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa167ed-f158-4b94-85d6-3151436c8dea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:09.505978 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa167ed-f158-4b94-85d6-3151436c8dea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:09.507519 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa167ed-f158-4b94-85d6-3151436c8dea None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:09.523418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-5aa167ed-f158-4b94-85d6-3151436c8dea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:18:09.523418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:18:09.523418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:18:09.523418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:18:09.523418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:18:09.523418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:18:09.523418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:18:09.523418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:18:09.523418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:18:09.523418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:18:09.523418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:18:09.523418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:18:09.523418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:18:09.523418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:18:09.523418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:18:09.523418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:18:09.524636 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:18:09.524636 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:18:09.524636 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:18:09.524636 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:18:09.524636 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:18:09.524636 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:18:09.524636 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:18:09.524636 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:18:09.524636 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:18:09.524636 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:18:09.524636 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:18:09.524636 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:18:09.524636 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:18:09.524636 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:18:09.524636 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:18:09.524636 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:18:09.524636 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:18:09.525605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:18:09.525605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:18:09.525605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:18:09.525605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 618/1237] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:18:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:18:09.530957 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15395334-0fd1-4ce1-8f50-2f6f547a9071 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:09.639496 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15395334-0fd1-4ce1-8f50-2f6f547a9071 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:09.649005 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:09.678610 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7082782a-cf1e-4cb7-b1b5-2d8cb47a12f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:09.679224 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082782a-cf1e-4cb7-b1b5-2d8cb47a12f8 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:09.679506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082782a-cf1e-4cb7-b1b5-2d8cb47a12f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:09.679778 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082782a-cf1e-4cb7-b1b5-2d8cb47a12f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:09.680421 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7082782a-cf1e-4cb7-b1b5-2d8cb47a12f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:09.680754 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7082782a-cf1e-4cb7-b1b5-2d8cb47a12f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:09.682120 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7082782a-cf1e-4cb7-b1b5-2d8cb47a12f8 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:09.694750 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 620/1238] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:18:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:18:09.700499 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa167ed-f158-4b94-85d6-3151436c8dea None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:09.810573 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa167ed-f158-4b94-85d6-3151436c8dea None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:09.850923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a248a950-969f-49c1-b8cc-0d560af6ab00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:09.851558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248a950-969f-49c1-b8cc-0d560af6ab00 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:09.851840 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248a950-969f-49c1-b8cc-0d560af6ab00 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:09.852167 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248a950-969f-49c1-b8cc-0d560af6ab00 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:09.855180 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:09.861769 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a248a950-969f-49c1-b8cc-0d560af6ab00 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:09.863541 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a248a950-969f-49c1-b8cc-0d560af6ab00 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:09.875801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-a248a950-969f-49c1-b8cc-0d560af6ab00 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 20:18:09.875801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:18:09.875801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:18:09.875801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:18:09.875801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:18:09.875801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:18:09.875801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:18:09.875801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:18:09.875801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:18:09.875801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:18:09.875801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:18:09.875801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:18:09.875801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:18:09.875801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:18:09.875801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:18:09.875801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:18:09.882307 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:18:09.882307 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:18:09.882307 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:18:09.882307 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:18:09.882307 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:18:09.882307 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:18:09.882307 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:18:09.882307 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:18:09.882307 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:18:09.882307 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:18:09.882307 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:18:09.882307 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:18:09.882307 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:18:09.882307 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:18:09.882307 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:18:09.882307 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:18:09.882307 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:18:09.882307 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:18:09.883548 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:18:09.883548 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:18:09.883548 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:18:09.883548 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:18:09.883548 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:18:09.883548 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:18:09.883548 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:18:09.883548 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:18:09.883548 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 20:18:09.883548 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:18:09.883548 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 619/1239] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:18:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:18:09.884323 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7082782a-cf1e-4cb7-b1b5-2d8cb47a12f8 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:09.994050 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7082782a-cf1e-4cb7-b1b5-2d8cb47a12f8 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:10.032233 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b82101-6806-410c-ba8d-3d7d5b69f0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:10.032855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b82101-6806-410c-ba8d-3d7d5b69f0a7 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:10.033133 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b82101-6806-410c-ba8d-3d7d5b69f0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:10.033441 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b82101-6806-410c-ba8d-3d7d5b69f0a7 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:10.034185 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b82101-6806-410c-ba8d-3d7d5b69f0a7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:10.034472 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b82101-6806-410c-ba8d-3d7d5b69f0a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:10.035924 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b82101-6806-410c-ba8d-3d7d5b69f0a7 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:10.061576 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 621/1240] 10.222.0.26 () {56 vars in 1143 bytes} [Fri Sep 9 20:18:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:18:10.069272 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a248a950-969f-49c1-b8cc-0d560af6ab00 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:10.178762 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a248a950-969f-49c1-b8cc-0d560af6ab00 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:10.214323 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f465d97e-b573-4b39-9440-e931724289cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:10.214921 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465d97e-b573-4b39-9440-e931724289cc None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:10.215197 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465d97e-b573-4b39-9440-e931724289cc None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:10.215465 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465d97e-b573-4b39-9440-e931724289cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:10.283814 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:10.291163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f465d97e-b573-4b39-9440-e931724289cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:10.292643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f465d97e-b573-4b39-9440-e931724289cc None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:10.402558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 620/1241] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:18:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:18:10.410262 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b82101-6806-410c-ba8d-3d7d5b69f0a7 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:10.448455 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:10.533509 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b82101-6806-410c-ba8d-3d7d5b69f0a7 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:10.570697 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcefe88-6d75-473d-8f17-a09b02a36ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:10.571244 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcefe88-6d75-473d-8f17-a09b02a36ebe None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:10.571527 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcefe88-6d75-473d-8f17-a09b02a36ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:10.571802 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcefe88-6d75-473d-8f17-a09b02a36ebe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:10.582825 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcefe88-6d75-473d-8f17-a09b02a36ebe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:10.584270 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcefe88-6d75-473d-8f17-a09b02a36ebe None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:10.599258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-bfcefe88-6d75-473d-8f17-a09b02a36ebe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:18:10.599258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:18:10.599258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:18:10.599258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:18:10.599258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:18:10.599258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:18:10.599258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:18:10.599258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:18:10.599258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:18:10.599258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:18:10.599258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:18:10.599258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:18:10.599258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:18:10.599258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:18:10.599258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:18:10.599258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:18:10.600461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:18:10.600461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:18:10.600461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:18:10.600461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:18:10.600461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:18:10.600461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:18:10.600461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:18:10.600461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:18:10.600461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:18:10.600461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:18:10.600461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:18:10.600461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:18:10.600461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:18:10.600461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:18:10.600461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:18:10.600461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:18:10.600461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:18:10.601416 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:18:10.601416 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:18:10.601416 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:18:10.601416 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 622/1242] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:18:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:18:10.606454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f465d97e-b573-4b39-9440-e931724289cc None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:10.715599 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:10.722555 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f465d97e-b573-4b39-9440-e931724289cc None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:10.758451 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ab2dbb-9757-4211-8cfe-f675415a52c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:10.759067 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab2dbb-9757-4211-8cfe-f675415a52c0 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:10.759382 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab2dbb-9757-4211-8cfe-f675415a52c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:10.759625 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab2dbb-9757-4211-8cfe-f675415a52c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:10.760266 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ab2dbb-9757-4211-8cfe-f675415a52c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:10.760569 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ab2dbb-9757-4211-8cfe-f675415a52c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:10.761961 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ab2dbb-9757-4211-8cfe-f675415a52c0 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:10.774175 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 621/1243] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:18:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:18:10.779974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcefe88-6d75-473d-8f17-a09b02a36ebe None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:10.869910 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:10.896245 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcefe88-6d75-473d-8f17-a09b02a36ebe None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:10.933048 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d1706-7866-4771-8dda-96f034aa2947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:10.933852 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d1706-7866-4771-8dda-96f034aa2947 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:10.934117 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d1706-7866-4771-8dda-96f034aa2947 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:10.934522 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d1706-7866-4771-8dda-96f034aa2947 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:10.943112 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d1706-7866-4771-8dda-96f034aa2947 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:10.945034 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d1706-7866-4771-8dda-96f034aa2947 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:10.956151 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-f35d1706-7866-4771-8dda-96f034aa2947 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:18:10.956151 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:18:10.956151 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:18:10.956151 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:18:10.956151 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:18:10.956151 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:18:10.956151 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:18:10.956151 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:18:10.956151 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:18:10.956151 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:18:10.956151 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:18:10.956151 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:18:10.956151 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:18:10.956151 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:18:10.956151 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:18:10.957332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:18:10.957332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:18:10.957332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:18:10.957332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:18:10.957332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:18:10.957332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:18:10.957332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:18:10.957332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:18:10.957332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:18:10.957332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:18:10.957332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:18:10.957332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:18:10.957332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:18:10.957332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:18:10.957332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:18:10.957332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:18:10.957332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:18:10.958538 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:18:10.958538 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:18:10.958538 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:18:10.958538 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:18:10.958538 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:18:10.958538 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:18:10.958538 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:18:10.958538 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:18:10.958538 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 623/1244] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:18:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:18:10.963937 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ab2dbb-9757-4211-8cfe-f675415a52c0 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:11.079399 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ab2dbb-9757-4211-8cfe-f675415a52c0 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:11.124389 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9931e019-129a-4188-a64e-e656cbc20997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:11.125481 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931e019-129a-4188-a64e-e656cbc20997 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:11.125846 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931e019-129a-4188-a64e-e656cbc20997 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:11.126215 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931e019-129a-4188-a64e-e656cbc20997 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:11.127082 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9931e019-129a-4188-a64e-e656cbc20997 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:11.127655 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9931e019-129a-4188-a64e-e656cbc20997 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:11.129203 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9931e019-129a-4188-a64e-e656cbc20997 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:11.144647 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 622/1245] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:18:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:18:11.151078 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d1706-7866-4771-8dda-96f034aa2947 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:11.264946 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d1706-7866-4771-8dda-96f034aa2947 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:11.300651 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62a383-a6ad-4569-8393-ad670bc7ed61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:11.301349 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62a383-a6ad-4569-8393-ad670bc7ed61 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:11.301650 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62a383-a6ad-4569-8393-ad670bc7ed61 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:11.301908 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62a383-a6ad-4569-8393-ad670bc7ed61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:11.302650 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a62a383-a6ad-4569-8393-ad670bc7ed61 None admin] RBAC: Adding query filter params (role.id=8ec32f702e0c447993ff294cbac536b1, scope.project.id=933820e2adbc473bbd57a4d59be75d72, user.id=26d24cf0a21d491082b0c2f11e749446) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:11.302937 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a62a383-a6ad-4569-8393-ad670bc7ed61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:11.304355 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a62a383-a6ad-4569-8393-ad670bc7ed61 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:11.310930 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 624/1246] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:18:11 2022] GET /identity/v3/role_assignments?user.id=26d24cf0a21d491082b0c2f11e749446&scope.project.id=933820e2adbc473bbd57a4d59be75d72&role.id=8ec32f702e0c447993ff294cbac536b1 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:18:13.639137 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3afd263-9add-4740-8024-6f075ce46468 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:13.639709 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3afd263-9add-4740-8024-6f075ce46468 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:13.639709 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3afd263-9add-4740-8024-6f075ce46468 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:13.641129 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 623/1247] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:18:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:18:13.646317 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5524b8-fb88-4acb-9d52-65d83bf00336 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:13.646769 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5524b8-fb88-4acb-9d52-65d83bf00336 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:13.646861 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5524b8-fb88-4acb-9d52-65d83bf00336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:13.653156 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:13.666995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-fb5524b8-fb88-4acb-9d52-65d83bf00336 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:18:13.668593 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-fb5524b8-fb88-4acb-9d52-65d83bf00336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:18:13.747983 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 625/1248] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:18:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:18:14.038223 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3afd263-9add-4740-8024-6f075ce46468 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:14.060424 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:14.154165 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3afd263-9add-4740-8024-6f075ce46468 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:14.191386 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124fbf93-2189-4cfb-8e86-bfff0e59ef11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:14.192017 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124fbf93-2189-4cfb-8e86-bfff0e59ef11 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:14.192305 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124fbf93-2189-4cfb-8e86-bfff0e59ef11 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:14.192582 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124fbf93-2189-4cfb-8e86-bfff0e59ef11 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:14.202818 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124fbf93-2189-4cfb-8e86-bfff0e59ef11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:18:14.207620 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-124fbf93-2189-4cfb-8e86-bfff0e59ef11 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:18:14.207620 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:18:14.207620 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:18:14.207620 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:18:14.207620 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:18:14.207620 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:18:14.207620 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:18:14.207620 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:18:14.207620 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:18:14.207620 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:18:14.207620 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:18:14.207620 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:18:14.207620 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:18:14.207620 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:18:14.207620 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:18:14.207620 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:18:14.209360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:18:14.209360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:18:14.209360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:18:14.209360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:18:14.209360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:18:14.209360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:18:14.209360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:18:14.209360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:18:14.209360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:18:14.209360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:18:14.209360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:18:14.209360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:18:14.209360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:18:14.209360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:18:14.209360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:18:14.209360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:18:14.211783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:18:14.211783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:18:14.211783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:18:14.211783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:18:14.211783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:18:14.211783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:18:14.211783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:18:14.211783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:18:14.211783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:18:14.211783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:18:14.211783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:18:14.211783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:18:14.211783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 624/1249] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:18:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:18:14.214835 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5524b8-fb88-4acb-9d52-65d83bf00336 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:14.328325 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5524b8-fb88-4acb-9d52-65d83bf00336 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:14.360344 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:14.366181 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d8112-043f-4b20-971a-3e025597da39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:14.366791 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d8112-043f-4b20-971a-3e025597da39 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:14.367167 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d8112-043f-4b20-971a-3e025597da39 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:14.367442 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d8112-043f-4b20-971a-3e025597da39 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:14.368101 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023d8112-043f-4b20-971a-3e025597da39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:14.368395 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023d8112-043f-4b20-971a-3e025597da39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:14.369807 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023d8112-043f-4b20-971a-3e025597da39 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:14.378540 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 626/1250] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:18:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:18:14.384799 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124fbf93-2189-4cfb-8e86-bfff0e59ef11 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:14.469617 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:14.494365 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124fbf93-2189-4cfb-8e86-bfff0e59ef11 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:14.530895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c44595-feb8-4601-a85d-7a246c125602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:14.531536 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c44595-feb8-4601-a85d-7a246c125602 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:14.531828 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c44595-feb8-4601-a85d-7a246c125602 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:14.532098 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c44595-feb8-4601-a85d-7a246c125602 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:14.543386 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c44595-feb8-4601-a85d-7a246c125602 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:14.544831 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c44595-feb8-4601-a85d-7a246c125602 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:14.561214 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-24c44595-feb8-4601-a85d-7a246c125602 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:18:14.561214 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:18:14.561214 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:18:14.561214 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:18:14.561214 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:18:14.561214 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:18:14.561214 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:18:14.561214 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:18:14.561214 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:18:14.561214 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:18:14.561214 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:18:14.561214 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:18:14.561214 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:18:14.561214 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:18:14.561214 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:18:14.561214 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:18:14.563091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:18:14.563091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:18:14.563091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:18:14.563091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:18:14.563091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:18:14.563091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:18:14.563091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:18:14.563091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:18:14.563091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:18:14.563091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:18:14.563091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:18:14.563091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:18:14.563091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:18:14.563091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:18:14.563091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:18:14.563091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:18:14.563091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:18:14.564092 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:18:14.564092 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:18:14.564092 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:18:14.564092 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 625/1251] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:18:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:18:14.569268 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d8112-043f-4b20-971a-3e025597da39 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:14.678007 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d8112-043f-4b20-971a-3e025597da39 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:14.715586 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a627c-cd84-4ea7-9aed-538ca83e8a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:14.716217 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a627c-cd84-4ea7-9aed-538ca83e8a34 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:14.716493 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a627c-cd84-4ea7-9aed-538ca83e8a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:14.716776 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a627c-cd84-4ea7-9aed-538ca83e8a34 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:14.717615 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a627c-cd84-4ea7-9aed-538ca83e8a34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:14.717925 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a627c-cd84-4ea7-9aed-538ca83e8a34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:14.719328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a627c-cd84-4ea7-9aed-538ca83e8a34 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:14.732706 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 627/1252] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:18:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:18:14.738621 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c44595-feb8-4601-a85d-7a246c125602 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:14.844940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c44595-feb8-4601-a85d-7a246c125602 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:14.871567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:14.880412 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2880d1b3-2b8d-4ba9-8ed6-cf44a2f2b240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:14.881059 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880d1b3-2b8d-4ba9-8ed6-cf44a2f2b240 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:14.881371 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880d1b3-2b8d-4ba9-8ed6-cf44a2f2b240 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:14.881648 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880d1b3-2b8d-4ba9-8ed6-cf44a2f2b240 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:14.890148 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2880d1b3-2b8d-4ba9-8ed6-cf44a2f2b240 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:14.891606 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2880d1b3-2b8d-4ba9-8ed6-cf44a2f2b240 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:14.904257 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-2880d1b3-2b8d-4ba9-8ed6-cf44a2f2b240 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 20:18:14.904257 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:18:14.904257 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:18:14.904257 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:18:14.904257 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:18:14.904257 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:18:14.904257 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:18:14.904257 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:18:14.904257 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:18:14.904257 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:18:14.904257 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:18:14.904257 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:18:14.904257 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:18:14.904257 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:18:14.904257 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:18:14.904257 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:18:14.905360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:18:14.905360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:18:14.905360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:18:14.905360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:18:14.905360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:18:14.905360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:18:14.905360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:18:14.905360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:18:14.905360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:18:14.905360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:18:14.905360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:18:14.905360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:18:14.905360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:18:14.905360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:18:14.905360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:18:14.905360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:18:14.905360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:18:14.905360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:18:14.906842 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:18:14.906842 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:18:14.906842 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:18:14.906842 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:18:14.906842 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:18:14.906842 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:18:14.906842 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:18:14.906842 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:18:14.906842 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 20:18:14.906842 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:18:14.906842 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 626/1253] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:18:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:18:14.911432 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a627c-cd84-4ea7-9aed-538ca83e8a34 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:14.969593 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:15.014265 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a627c-cd84-4ea7-9aed-538ca83e8a34 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:15.052281 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5309900e-ea2b-4348-9e21-9ae6bff50e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:15.052959 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309900e-ea2b-4348-9e21-9ae6bff50e96 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:15.053378 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309900e-ea2b-4348-9e21-9ae6bff50e96 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:15.053757 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309900e-ea2b-4348-9e21-9ae6bff50e96 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:15.054535 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5309900e-ea2b-4348-9e21-9ae6bff50e96 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:15.054850 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5309900e-ea2b-4348-9e21-9ae6bff50e96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:15.056334 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5309900e-ea2b-4348-9e21-9ae6bff50e96 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:15.077686 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 628/1254] 10.222.0.26 () {56 vars in 1143 bytes} [Fri Sep 9 20:18:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:18:15.084689 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2880d1b3-2b8d-4ba9-8ed6-cf44a2f2b240 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:15.202065 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2880d1b3-2b8d-4ba9-8ed6-cf44a2f2b240 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:15.235836 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122bfa54-877f-4a82-9eb1-de952873e490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:15.236903 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122bfa54-877f-4a82-9eb1-de952873e490 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:15.237182 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122bfa54-877f-4a82-9eb1-de952873e490 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:15.237495 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122bfa54-877f-4a82-9eb1-de952873e490 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:15.297793 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:15.310801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122bfa54-877f-4a82-9eb1-de952873e490 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:15.312237 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122bfa54-877f-4a82-9eb1-de952873e490 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:15.406971 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 627/1255] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:18:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:18:15.412989 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5309900e-ea2b-4348-9e21-9ae6bff50e96 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:15.509629 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5309900e-ea2b-4348-9e21-9ae6bff50e96 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:15.523355 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:15.548303 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a50c188-a148-437e-bdae-e946adfbbae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:15.549633 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50c188-a148-437e-bdae-e946adfbbae4 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:15.549937 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50c188-a148-437e-bdae-e946adfbbae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:15.550210 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50c188-a148-437e-bdae-e946adfbbae4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:15.560338 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a50c188-a148-437e-bdae-e946adfbbae4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:15.561796 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a50c188-a148-437e-bdae-e946adfbbae4 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:15.575705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-8a50c188-a148-437e-bdae-e946adfbbae4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:18:15.575705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:18:15.575705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:18:15.575705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:18:15.575705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:18:15.575705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:18:15.575705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:18:15.575705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:18:15.575705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:18:15.575705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:18:15.575705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:18:15.575705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:18:15.575705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:18:15.575705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:18:15.575705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:18:15.575705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:18:15.580585 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:18:15.580585 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:18:15.580585 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:18:15.580585 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:18:15.580585 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:18:15.580585 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:18:15.580585 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:18:15.580585 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:18:15.580585 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:18:15.580585 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:18:15.580585 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:18:15.580585 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:18:15.580585 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:18:15.580585 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:18:15.580585 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:18:15.580585 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:18:15.580585 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:18:15.581768 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:18:15.581768 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:18:15.581768 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:18:15.581768 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 629/1256] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:18:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:18:15.582712 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122bfa54-877f-4a82-9eb1-de952873e490 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:15.666385 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:15.679185 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122bfa54-877f-4a82-9eb1-de952873e490 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:15.713024 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83095e9-fe62-402f-94e1-00e6daf476a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:15.713648 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83095e9-fe62-402f-94e1-00e6daf476a6 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:15.713929 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83095e9-fe62-402f-94e1-00e6daf476a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:15.714207 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83095e9-fe62-402f-94e1-00e6daf476a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:15.714842 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83095e9-fe62-402f-94e1-00e6daf476a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:15.715166 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83095e9-fe62-402f-94e1-00e6daf476a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:15.716490 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83095e9-fe62-402f-94e1-00e6daf476a6 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:15.727559 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 628/1257] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:18:15.733280 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a50c188-a148-437e-bdae-e946adfbbae4 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:15.835924 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a50c188-a148-437e-bdae-e946adfbbae4 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:15.869074 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d259de-36a0-4085-9bb3-38ac7a65f79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:15.869716 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d259de-36a0-4085-9bb3-38ac7a65f79c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:15.869991 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d259de-36a0-4085-9bb3-38ac7a65f79c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:15.870260 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d259de-36a0-4085-9bb3-38ac7a65f79c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:15.876674 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d259de-36a0-4085-9bb3-38ac7a65f79c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:15.878642 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d259de-36a0-4085-9bb3-38ac7a65f79c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:15.889582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-c4d259de-36a0-4085-9bb3-38ac7a65f79c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:18:15.889582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:18:15.889582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:18:15.889582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:18:15.889582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:18:15.889582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:18:15.889582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:18:15.889582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:18:15.889582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:18:15.889582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:18:15.889582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:18:15.889582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:18:15.889582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:18:15.889582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:18:15.889582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:18:15.890656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:18:15.890656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:18:15.890656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:18:15.890656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:18:15.890656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:18:15.890656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:18:15.890656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:18:15.890656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:18:15.890656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:18:15.890656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:18:15.890656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:18:15.890656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:18:15.890656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:18:15.890656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:18:15.890656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:18:15.890656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:18:15.890656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:18:15.891589 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:18:15.891589 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:18:15.891589 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:18:15.891589 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:18:15.891589 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:18:15.891589 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:18:15.891589 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:18:15.891589 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:18:15.891589 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 630/1258] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:18:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:18:15.897192 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83095e9-fe62-402f-94e1-00e6daf476a6 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:15.994120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83095e9-fe62-402f-94e1-00e6daf476a6 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:16.026764 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70524328-5820-4317-8d04-efddc0250585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:16.027359 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70524328-5820-4317-8d04-efddc0250585 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:16.027641 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70524328-5820-4317-8d04-efddc0250585 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:16.027915 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70524328-5820-4317-8d04-efddc0250585 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:16.028596 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70524328-5820-4317-8d04-efddc0250585 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:16.028900 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70524328-5820-4317-8d04-efddc0250585 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:16.030344 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70524328-5820-4317-8d04-efddc0250585 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:16.042590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 629/1259] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:18:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:18:16.048909 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d259de-36a0-4085-9bb3-38ac7a65f79c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:16.075267 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:16.151909 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d259de-36a0-4085-9bb3-38ac7a65f79c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:16.184762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c162ea9f-314d-4b6a-b381-19a07f49453a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:16.185402 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c162ea9f-314d-4b6a-b381-19a07f49453a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:16.185682 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c162ea9f-314d-4b6a-b381-19a07f49453a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:16.185954 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c162ea9f-314d-4b6a-b381-19a07f49453a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:16.186671 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c162ea9f-314d-4b6a-b381-19a07f49453a None admin] RBAC: Adding query filter params (role.id=9121ba25a6d84756990f0c3a19e9fecb, scope.project.id=933820e2adbc473bbd57a4d59be75d72, user.id=26d24cf0a21d491082b0c2f11e749446) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:16.186977 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c162ea9f-314d-4b6a-b381-19a07f49453a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:16.188394 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c162ea9f-314d-4b6a-b381-19a07f49453a None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:16.194165 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 631/1260] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:18:16 2022] GET /identity/v3/role_assignments?user.id=26d24cf0a21d491082b0c2f11e749446&scope.project.id=933820e2adbc473bbd57a4d59be75d72&role.id=9121ba25a6d84756990f0c3a19e9fecb => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:18:18.324106 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c422665-5208-4f37-8f1c-9b255baca5c9 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:18.324643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c422665-5208-4f37-8f1c-9b255baca5c9 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:18.324643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c422665-5208-4f37-8f1c-9b255baca5c9 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:18.326015 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 630/1261] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:18:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:18:18.331185 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c0e72-4970-40b9-b337-6335cf860144 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:18.331621 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c0e72-4970-40b9-b337-6335cf860144 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:18.331714 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c0e72-4970-40b9-b337-6335cf860144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:18.349440 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-6c3c0e72-4970-40b9-b337-6335cf860144 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:18:18.350619 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6c3c0e72-4970-40b9-b337-6335cf860144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:18:18.414839 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 632/1262] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:18:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:18:18.684833 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c422665-5208-4f37-8f1c-9b255baca5c9 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:18.695961 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:18.785434 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c422665-5208-4f37-8f1c-9b255baca5c9 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:18.818080 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8aeea-b913-4b03-af63-c60c0fa7aec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:18.818696 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8aeea-b913-4b03-af63-c60c0fa7aec2 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:18.818979 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8aeea-b913-4b03-af63-c60c0fa7aec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:18.819253 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8aeea-b913-4b03-af63-c60c0fa7aec2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:18.829020 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f8aeea-b913-4b03-af63-c60c0fa7aec2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:18:18.834315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-67f8aeea-b913-4b03-af63-c60c0fa7aec2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:18:18.834315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:18:18.834315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:18:18.834315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:18:18.834315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:18:18.834315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:18:18.834315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:18:18.834315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:18:18.834315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:18:18.834315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:18:18.834315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:18:18.834315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:18:18.834315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:18:18.834315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:18:18.834315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:18:18.834315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:18:18.835403 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:18:18.835403 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:18:18.835403 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:18:18.835403 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:18:18.835403 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:18:18.835403 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:18:18.835403 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:18:18.835403 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:18:18.835403 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:18:18.835403 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:18:18.835403 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:18:18.835403 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:18:18.835403 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:18:18.835403 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:18:18.835403 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:18:18.835403 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:18:18.836470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:18:18.836470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:18:18.836470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:18:18.836470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:18:18.836470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:18:18.836470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:18:18.836470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:18:18.836470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:18:18.836470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:18:18.836470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:18:18.836470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:18:18.836470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:18:18.836470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 631/1263] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:18:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:18:18.841612 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c0e72-4970-40b9-b337-6335cf860144 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:18.859962 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:18.938293 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c0e72-4970-40b9-b337-6335cf860144 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:18.971357 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ce97f-9b7e-413d-8ba6-42dabe6f8587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:18.971820 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ce97f-9b7e-413d-8ba6-42dabe6f8587 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:18.972096 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ce97f-9b7e-413d-8ba6-42dabe6f8587 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:18.972363 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ce97f-9b7e-413d-8ba6-42dabe6f8587 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:18.973017 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ce97f-9b7e-413d-8ba6-42dabe6f8587 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:18.973343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ce97f-9b7e-413d-8ba6-42dabe6f8587 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:18.974691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ce97f-9b7e-413d-8ba6-42dabe6f8587 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:18.982490 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 633/1264] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:18:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:18:18.988173 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8aeea-b913-4b03-af63-c60c0fa7aec2 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:19.057945 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:19.098072 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8aeea-b913-4b03-af63-c60c0fa7aec2 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:19.131778 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c4959-c607-435d-8972-396e66c6a612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:19.132405 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c4959-c607-435d-8972-396e66c6a612 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:19.132688 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c4959-c607-435d-8972-396e66c6a612 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:19.132964 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c4959-c607-435d-8972-396e66c6a612 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:19.143245 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c4959-c607-435d-8972-396e66c6a612 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:19.144663 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c4959-c607-435d-8972-396e66c6a612 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:19.159373 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-e31c4959-c607-435d-8972-396e66c6a612 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:18:19.159373 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:18:19.159373 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:18:19.159373 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:18:19.159373 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:18:19.159373 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:18:19.159373 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:18:19.159373 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:18:19.159373 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:18:19.159373 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:18:19.159373 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:18:19.159373 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:18:19.159373 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:18:19.159373 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:18:19.159373 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:18:19.159373 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:18:19.160476 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:18:19.160476 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:18:19.160476 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:18:19.160476 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:18:19.160476 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:18:19.160476 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:18:19.160476 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:18:19.160476 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:18:19.160476 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:18:19.160476 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:18:19.160476 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:18:19.160476 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:18:19.160476 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:18:19.160476 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:18:19.160476 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:18:19.160476 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:18:19.160476 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:18:19.161414 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:18:19.161414 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:18:19.161414 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:18:19.161414 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 632/1265] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:18:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:18:19.166236 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ce97f-9b7e-413d-8ba6-42dabe6f8587 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:19.246071 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:19.265276 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ce97f-9b7e-413d-8ba6-42dabe6f8587 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:19.297701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccdda07-afbe-4524-8930-2562b8aee930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:19.298292 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdda07-afbe-4524-8930-2562b8aee930 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:19.298619 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdda07-afbe-4524-8930-2562b8aee930 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:19.299394 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdda07-afbe-4524-8930-2562b8aee930 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:19.300029 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccdda07-afbe-4524-8930-2562b8aee930 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:19.300330 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccdda07-afbe-4524-8930-2562b8aee930 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:19.301712 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccdda07-afbe-4524-8930-2562b8aee930 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:19.313028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 634/1266] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:18:19.318644 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c4959-c607-435d-8972-396e66c6a612 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:19.414917 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c4959-c607-435d-8972-396e66c6a612 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:19.434009 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:19.447760 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155fe00e-c745-4ec7-bc88-f80b65e058e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:19.448378 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155fe00e-c745-4ec7-bc88-f80b65e058e0 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:19.448661 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155fe00e-c745-4ec7-bc88-f80b65e058e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:19.448936 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155fe00e-c745-4ec7-bc88-f80b65e058e0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:19.456196 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155fe00e-c745-4ec7-bc88-f80b65e058e0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:19.457635 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155fe00e-c745-4ec7-bc88-f80b65e058e0 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:19.468378 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-155fe00e-c745-4ec7-bc88-f80b65e058e0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 20:18:19.468378 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:18:19.468378 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:18:19.468378 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:18:19.468378 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:18:19.468378 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:18:19.468378 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:18:19.468378 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:18:19.468378 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:18:19.468378 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:18:19.468378 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:18:19.468378 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:18:19.468378 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:18:19.468378 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:18:19.468378 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:18:19.468378 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:18:19.469643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:18:19.469643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:18:19.469643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:18:19.469643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:18:19.469643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:18:19.469643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:18:19.469643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:18:19.469643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:18:19.469643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:18:19.469643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:18:19.469643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:18:19.469643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:18:19.469643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:18:19.469643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:18:19.469643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:18:19.469643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:18:19.469643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:18:19.469643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:18:19.471321 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:18:19.471321 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:18:19.471321 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:18:19.471321 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:18:19.471321 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:18:19.471321 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:18:19.471321 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:18:19.471321 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:18:19.471321 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 20:18:19.471321 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:18:19.471321 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 633/1267] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:18:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:18:19.475764 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccdda07-afbe-4524-8930-2562b8aee930 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:19.572478 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccdda07-afbe-4524-8930-2562b8aee930 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:19.605344 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b37e05-1b15-4958-a8ef-fc10a95293eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:19.605930 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b37e05-1b15-4958-a8ef-fc10a95293eb None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:19.606205 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b37e05-1b15-4958-a8ef-fc10a95293eb None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:19.606473 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b37e05-1b15-4958-a8ef-fc10a95293eb None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:19.607182 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b37e05-1b15-4958-a8ef-fc10a95293eb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:19.607480 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b37e05-1b15-4958-a8ef-fc10a95293eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:19.608887 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b37e05-1b15-4958-a8ef-fc10a95293eb None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:19.617401 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:19.626028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 635/1268] 10.222.0.26 () {56 vars in 1143 bytes} [Fri Sep 9 20:18:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:18:19.632029 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155fe00e-c745-4ec7-bc88-f80b65e058e0 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:19.728892 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155fe00e-c745-4ec7-bc88-f80b65e058e0 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:19.760637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77cf240-7101-4111-826c-ed44b7c1aade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:19.761280 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77cf240-7101-4111-826c-ed44b7c1aade None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:19.761561 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77cf240-7101-4111-826c-ed44b7c1aade None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:19.761834 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77cf240-7101-4111-826c-ed44b7c1aade None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:19.772388 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77cf240-7101-4111-826c-ed44b7c1aade None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:19.773838 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77cf240-7101-4111-826c-ed44b7c1aade None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:19.787402 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-b77cf240-7101-4111-826c-ed44b7c1aade None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:18:19.787402 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:18:19.787402 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:18:19.787402 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:18:19.787402 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:18:19.787402 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:18:19.787402 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:18:19.787402 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:18:19.787402 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:18:19.787402 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:18:19.787402 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:18:19.787402 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:18:19.787402 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:18:19.787402 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:18:19.787402 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:18:19.787402 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:18:19.788552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:18:19.788552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:18:19.788552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:18:19.788552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:18:19.788552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:18:19.788552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:18:19.788552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:18:19.788552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:18:19.788552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:18:19.788552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:18:19.788552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:18:19.788552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:18:19.788552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:18:19.788552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:18:19.788552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:18:19.788552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:18:19.788552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:18:19.789693 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:18:19.789693 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:18:19.789693 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:18:19.789693 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 634/1269] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:18:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:18:19.794164 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b37e05-1b15-4958-a8ef-fc10a95293eb None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:19.890454 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b37e05-1b15-4958-a8ef-fc10a95293eb None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:19.922370 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d306ef1-ade2-4aa6-b099-ff5b159c395e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:19.922962 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d306ef1-ade2-4aa6-b099-ff5b159c395e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:19.923239 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d306ef1-ade2-4aa6-b099-ff5b159c395e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:19.924015 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d306ef1-ade2-4aa6-b099-ff5b159c395e None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:19.924660 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d306ef1-ade2-4aa6-b099-ff5b159c395e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:19.924957 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d306ef1-ade2-4aa6-b099-ff5b159c395e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:19.926345 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d306ef1-ade2-4aa6-b099-ff5b159c395e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:19.937102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 636/1270] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:18:19.942765 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77cf240-7101-4111-826c-ed44b7c1aade None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:19.973706 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:20.040026 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77cf240-7101-4111-826c-ed44b7c1aade None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:20.076813 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ccc70e-4c79-49e7-a06c-f083fb53cd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:20.077475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ccc70e-4c79-49e7-a06c-f083fb53cd73 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:20.077759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ccc70e-4c79-49e7-a06c-f083fb53cd73 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:20.078032 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ccc70e-4c79-49e7-a06c-f083fb53cd73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:20.084697 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ccc70e-4c79-49e7-a06c-f083fb53cd73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:20.086156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ccc70e-4c79-49e7-a06c-f083fb53cd73 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:20.097642 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-87ccc70e-4c79-49e7-a06c-f083fb53cd73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:18:20.097642 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:18:20.097642 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:18:20.097642 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:18:20.097642 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:18:20.097642 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:18:20.097642 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:18:20.097642 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:18:20.097642 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:18:20.097642 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:18:20.097642 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:18:20.097642 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:18:20.097642 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:18:20.097642 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:18:20.097642 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:18:20.098754 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:18:20.098754 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:18:20.098754 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:18:20.098754 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:18:20.098754 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:18:20.098754 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:18:20.098754 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:18:20.098754 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:18:20.098754 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:18:20.098754 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:18:20.098754 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:18:20.098754 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:18:20.098754 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:18:20.098754 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:18:20.098754 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:18:20.098754 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:18:20.098754 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:18:20.099711 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:18:20.099711 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:18:20.099711 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:18:20.099711 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:18:20.099711 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:18:20.099711 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:18:20.099711 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:18:20.099711 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:18:20.099711 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 635/1271] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:18:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:18:20.104556 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d306ef1-ade2-4aa6-b099-ff5b159c395e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:20.161554 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:20.201997 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d306ef1-ade2-4aa6-b099-ff5b159c395e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:20.234305 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41e8b30-39f7-4476-b842-c2102d7f88a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:20.234901 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e8b30-39f7-4476-b842-c2102d7f88a4 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:20.235182 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e8b30-39f7-4476-b842-c2102d7f88a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:20.235457 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e8b30-39f7-4476-b842-c2102d7f88a4 None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:20.236693 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41e8b30-39f7-4476-b842-c2102d7f88a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:20.237001 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41e8b30-39f7-4476-b842-c2102d7f88a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:20.238452 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41e8b30-39f7-4476-b842-c2102d7f88a4 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:20.249934 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 637/1272] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:18:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:18:20.256082 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ccc70e-4c79-49e7-a06c-f083fb53cd73 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:20.343186 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:20.352464 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ccc70e-4c79-49e7-a06c-f083fb53cd73 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:20.384152 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759adc5b-80f0-41bb-8f83-86c583857eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:20.384796 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759adc5b-80f0-41bb-8f83-86c583857eba None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:20.385081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759adc5b-80f0-41bb-8f83-86c583857eba None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:20.385384 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759adc5b-80f0-41bb-8f83-86c583857eba None admin] PATH_INFO: `/v3/projects/933820e2adbc473bbd57a4d59be75d72/users/26d24cf0a21d491082b0c2f11e749446/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:20.387225 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759adc5b-80f0-41bb-8f83-86c583857eba None admin] RBAC: Authorizing `identity:create_grant(project_id=933820e2adbc473bbd57a4d59be75d72, user_id=26d24cf0a21d491082b0c2f11e749446, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:20.388612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759adc5b-80f0-41bb-8f83-86c583857eba None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:20.400689 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 636/1273] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Sep 9 20:18:20 2022] PUT /identity/v3/projects/933820e2adbc473bbd57a4d59be75d72/users/26d24cf0a21d491082b0c2f11e749446/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:18:22.513037 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f97c0-92d2-4d6d-86cb-f41ffbaa594b None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:22.513598 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f97c0-92d2-4d6d-86cb-f41ffbaa594b None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:22.513598 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f97c0-92d2-4d6d-86cb-f41ffbaa594b None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:22.514928 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 638/1274] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:18:22.520048 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb6a32-8519-4b78-94d4-dfe46a0520c4 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:22.520456 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb6a32-8519-4b78-94d4-dfe46a0520c4 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:22.520602 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb6a32-8519-4b78-94d4-dfe46a0520c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:22.538798 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e2fb6a32-8519-4b78-94d4-dfe46a0520c4 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:18:22.539964 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e2fb6a32-8519-4b78-94d4-dfe46a0520c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:18:22.609162 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 637/1275] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:18:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:18:22.882164 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f97c0-92d2-4d6d-86cb-f41ffbaa594b None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:22.980263 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f97c0-92d2-4d6d-86cb-f41ffbaa594b None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:22.997620 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:23.014773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f4886f-3f2c-420d-8cc8-cd6d42fc4af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:23.015389 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f4886f-3f2c-420d-8cc8-cd6d42fc4af7 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:23.015669 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f4886f-3f2c-420d-8cc8-cd6d42fc4af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:23.015942 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f4886f-3f2c-420d-8cc8-cd6d42fc4af7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:23.026336 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f4886f-3f2c-420d-8cc8-cd6d42fc4af7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:18:23.031051 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-08f4886f-3f2c-420d-8cc8-cd6d42fc4af7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:18:23.031051 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:18:23.031051 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:18:23.031051 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:18:23.031051 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:18:23.031051 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:18:23.031051 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:18:23.031051 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:18:23.031051 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:18:23.031051 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:18:23.031051 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:18:23.031051 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:18:23.031051 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:18:23.031051 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:18:23.031051 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:18:23.031051 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:18:23.032157 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:18:23.032157 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:18:23.032157 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:18:23.032157 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:18:23.032157 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:18:23.032157 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:18:23.032157 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:18:23.032157 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:18:23.032157 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:18:23.032157 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:18:23.032157 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:18:23.032157 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:18:23.032157 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:18:23.032157 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:18:23.032157 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:18:23.032157 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:18:23.033054 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:18:23.033054 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:18:23.033054 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:18:23.033054 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:18:23.033054 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:18:23.033054 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:18:23.033054 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:18:23.033054 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:18:23.033054 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:18:23.033054 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:18:23.033054 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:18:23.033054 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:18:23.033054 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 639/1276] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Sep 9 20:18:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:18:23.038131 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb6a32-8519-4b78-94d4-dfe46a0520c4 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:23.128847 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:23.147881 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb6a32-8519-4b78-94d4-dfe46a0520c4 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:23.180521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da4f882-cb44-47e0-b25a-71ac98eceacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:23.181120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4f882-cb44-47e0-b25a-71ac98eceacd None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:23.181427 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4f882-cb44-47e0-b25a-71ac98eceacd None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:23.181699 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4f882-cb44-47e0-b25a-71ac98eceacd None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:23.182354 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da4f882-cb44-47e0-b25a-71ac98eceacd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:23.182651 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da4f882-cb44-47e0-b25a-71ac98eceacd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:23.183997 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da4f882-cb44-47e0-b25a-71ac98eceacd None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:23.192175 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 638/1277] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Sep 9 20:18:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:18:23.198081 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f4886f-3f2c-420d-8cc8-cd6d42fc4af7 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:23.294774 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f4886f-3f2c-420d-8cc8-cd6d42fc4af7 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:23.327965 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f81c5-6b8a-4a50-99c2-5eb92a4b6e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:23.328582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f81c5-6b8a-4a50-99c2-5eb92a4b6e1a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:23.328863 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f81c5-6b8a-4a50-99c2-5eb92a4b6e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:23.329137 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f81c5-6b8a-4a50-99c2-5eb92a4b6e1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:23.339317 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2f81c5-6b8a-4a50-99c2-5eb92a4b6e1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:23.340745 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2f81c5-6b8a-4a50-99c2-5eb92a4b6e1a None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:23.354341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-7c2f81c5-6b8a-4a50-99c2-5eb92a4b6e1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:18:23.354341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:18:23.354341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:18:23.354341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:18:23.354341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:18:23.354341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:18:23.354341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:18:23.354341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:18:23.354341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:18:23.354341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:18:23.354341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:18:23.354341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:18:23.354341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:18:23.354341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:18:23.354341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:18:23.354341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:18:23.355504 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:18:23.355504 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:18:23.355504 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:18:23.355504 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:18:23.355504 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:18:23.355504 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:18:23.355504 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:18:23.355504 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:18:23.355504 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:18:23.355504 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:18:23.355504 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:18:23.355504 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:18:23.355504 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:18:23.355504 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:18:23.355504 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:18:23.355504 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:18:23.355504 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:18:23.356653 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:18:23.356653 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:18:23.356653 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:18:23.356653 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 640/1278] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:18:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:18:23.361592 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da4f882-cb44-47e0-b25a-71ac98eceacd None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:23.457730 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da4f882-cb44-47e0-b25a-71ac98eceacd None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:23.489657 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e7b27-42fa-4f9e-93c4-7a83dd075c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:23.490248 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e7b27-42fa-4f9e-93c4-7a83dd075c20 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:23.490527 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e7b27-42fa-4f9e-93c4-7a83dd075c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:23.490799 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e7b27-42fa-4f9e-93c4-7a83dd075c20 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:23.491430 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33e7b27-42fa-4f9e-93c4-7a83dd075c20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:23.491728 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33e7b27-42fa-4f9e-93c4-7a83dd075c20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:23.493059 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33e7b27-42fa-4f9e-93c4-7a83dd075c20 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:23.505031 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 639/1279] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:18:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:18:23.510580 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f81c5-6b8a-4a50-99c2-5eb92a4b6e1a None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:23.540116 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:23.607384 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f81c5-6b8a-4a50-99c2-5eb92a4b6e1a None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:23.640303 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44bc3de-843f-47c3-98a4-de6b677f8fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:23.640914 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bc3de-843f-47c3-98a4-de6b677f8fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:23.641194 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bc3de-843f-47c3-98a4-de6b677f8fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:23.641496 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bc3de-843f-47c3-98a4-de6b677f8fc4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:23.648583 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44bc3de-843f-47c3-98a4-de6b677f8fc4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:23.650041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44bc3de-843f-47c3-98a4-de6b677f8fc4 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:23.660928 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-e44bc3de-843f-47c3-98a4-de6b677f8fc4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 20:18:23.660928 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:18:23.660928 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:18:23.660928 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:18:23.660928 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:18:23.660928 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:18:23.660928 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:18:23.660928 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:18:23.660928 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:18:23.660928 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:18:23.660928 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:18:23.660928 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:18:23.660928 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:18:23.660928 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:18:23.660928 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:18:23.660928 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:18:23.662115 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:18:23.662115 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:18:23.662115 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:18:23.662115 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:18:23.662115 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:18:23.662115 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:18:23.662115 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:18:23.662115 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:18:23.662115 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:18:23.662115 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:18:23.662115 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:18:23.662115 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:18:23.662115 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:18:23.662115 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:18:23.662115 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:18:23.662115 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:18:23.662115 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:18:23.662115 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:18:23.663341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:18:23.663341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:18:23.663341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:18:23.663341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:18:23.663341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:18:23.663341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:18:23.663341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:18:23.663341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:18:23.663341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 20:18:23.663341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:18:23.663341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 641/1280] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:18:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:18:23.667880 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e7b27-42fa-4f9e-93c4-7a83dd075c20 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:23.671719 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:23.764800 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e7b27-42fa-4f9e-93c4-7a83dd075c20 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:23.797086 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a335e8f-f36b-4dde-afb5-860ea9c513b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:23.797706 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a335e8f-f36b-4dde-afb5-860ea9c513b8 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:23.797989 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a335e8f-f36b-4dde-afb5-860ea9c513b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:23.798259 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a335e8f-f36b-4dde-afb5-860ea9c513b8 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:23.799545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a335e8f-f36b-4dde-afb5-860ea9c513b8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:23.799846 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a335e8f-f36b-4dde-afb5-860ea9c513b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:23.801266 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a335e8f-f36b-4dde-afb5-860ea9c513b8 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:23.817436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 640/1281] 10.222.0.26 () {56 vars in 1143 bytes} [Fri Sep 9 20:18:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:18:23.823237 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44bc3de-843f-47c3-98a4-de6b677f8fc4 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:23.916697 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:23.926488 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44bc3de-843f-47c3-98a4-de6b677f8fc4 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:23.959280 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c67151-78b8-4e92-86fa-81aa67f0447c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:23.959874 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c67151-78b8-4e92-86fa-81aa67f0447c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:23.960148 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c67151-78b8-4e92-86fa-81aa67f0447c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:23.960418 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c67151-78b8-4e92-86fa-81aa67f0447c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:24.025115 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c67151-78b8-4e92-86fa-81aa67f0447c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:24.026526 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c67151-78b8-4e92-86fa-81aa67f0447c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:24.115325 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:24.125509 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 642/1282] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Sep 9 20:18:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:18:24.132273 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a335e8f-f36b-4dde-afb5-860ea9c513b8 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:24.191100 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:24.227610 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a335e8f-f36b-4dde-afb5-860ea9c513b8 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:24.260678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d0113-fce4-4229-a260-6c7c354a0f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:24.261313 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d0113-fce4-4229-a260-6c7c354a0f2c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:24.261594 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d0113-fce4-4229-a260-6c7c354a0f2c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:24.261863 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d0113-fce4-4229-a260-6c7c354a0f2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:24.271446 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6d0113-fce4-4229-a260-6c7c354a0f2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:24.272857 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6d0113-fce4-4229-a260-6c7c354a0f2c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:24.286923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-8b6d0113-fce4-4229-a260-6c7c354a0f2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:18:24.286923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:18:24.286923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:18:24.286923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:18:24.286923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:18:24.286923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:18:24.286923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:18:24.286923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:18:24.286923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:18:24.286923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:18:24.286923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:18:24.286923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:18:24.286923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:18:24.286923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:18:24.286923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:18:24.286923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:18:24.287904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:18:24.287904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:18:24.287904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:18:24.287904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:18:24.287904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:18:24.287904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:18:24.287904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:18:24.287904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:18:24.287904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:18:24.287904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:18:24.287904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:18:24.287904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:18:24.287904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:18:24.287904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:18:24.287904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:18:24.287904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:18:24.287904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:18:24.288987 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:18:24.288987 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:18:24.288987 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:18:24.288987 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 641/1283] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:18:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:18:24.293681 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c67151-78b8-4e92-86fa-81aa67f0447c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:24.388786 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c67151-78b8-4e92-86fa-81aa67f0447c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:24.420405 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f3a77-b197-4324-a180-8259b17f5d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:24.420987 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f3a77-b197-4324-a180-8259b17f5d7d None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:24.421279 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f3a77-b197-4324-a180-8259b17f5d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:24.421546 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f3a77-b197-4324-a180-8259b17f5d7d None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:24.422166 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f3a77-b197-4324-a180-8259b17f5d7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:24.422461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f3a77-b197-4324-a180-8259b17f5d7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:24.424463 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f3a77-b197-4324-a180-8259b17f5d7d None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:24.435248 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 643/1284] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Sep 9 20:18:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:18:24.441306 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d0113-fce4-4229-a260-6c7c354a0f2c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:24.538664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d0113-fce4-4229-a260-6c7c354a0f2c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:24.552009 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:24.571446 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a094dab-f062-4c7b-9986-352291f0310f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:24.572065 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a094dab-f062-4c7b-9986-352291f0310f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:24.572345 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a094dab-f062-4c7b-9986-352291f0310f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:24.572615 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a094dab-f062-4c7b-9986-352291f0310f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:24.578945 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a094dab-f062-4c7b-9986-352291f0310f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:24.580380 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a094dab-f062-4c7b-9986-352291f0310f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:24.591238 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-8a094dab-f062-4c7b-9986-352291f0310f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:18:24.591238 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:18:24.591238 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:18:24.591238 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:18:24.591238 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:18:24.591238 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:18:24.591238 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:18:24.591238 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:18:24.591238 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:18:24.591238 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:18:24.591238 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:18:24.591238 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:18:24.591238 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:18:24.591238 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:18:24.591238 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:18:24.592147 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:18:24.592147 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:18:24.592147 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:18:24.592147 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:18:24.592147 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:18:24.592147 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:18:24.592147 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:18:24.592147 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:18:24.592147 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 09 20:18:24.592147 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:18:24.592147 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:18:24.592147 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:18:24.592147 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:18:24.592147 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:18:24.592147 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:18:24.592147 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:18:24.592147 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:18:24.593146 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:18:24.593146 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:18:24.593146 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:18:24.593146 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:18:24.593146 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:18:24.593146 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:18:24.593146 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:18:24.593146 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 09 20:18:24.593146 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 642/1285] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Sep 9 20:18:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:18:24.598744 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f3a77-b197-4324-a180-8259b17f5d7d None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:24.647033 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:24.699566 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f3a77-b197-4324-a180-8259b17f5d7d None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:24.732093 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49c1131-4dfc-4179-bda8-e461fc0b2ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:24.732681 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c1131-4dfc-4179-bda8-e461fc0b2ddf None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:24.732952 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c1131-4dfc-4179-bda8-e461fc0b2ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:24.733215 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c1131-4dfc-4179-bda8-e461fc0b2ddf None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:24.733918 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49c1131-4dfc-4179-bda8-e461fc0b2ddf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:24.734214 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49c1131-4dfc-4179-bda8-e461fc0b2ddf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:24.735612 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49c1131-4dfc-4179-bda8-e461fc0b2ddf None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:24.747094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 644/1286] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Sep 9 20:18:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:18:24.753978 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a094dab-f062-4c7b-9986-352291f0310f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:24.851657 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a094dab-f062-4c7b-9986-352291f0310f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:24.884869 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977246c3-f5a5-46c0-8388-af8b9044fc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:24.885496 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977246c3-f5a5-46c0-8388-af8b9044fc11 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:24.885780 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977246c3-f5a5-46c0-8388-af8b9044fc11 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:24.886055 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977246c3-f5a5-46c0-8388-af8b9044fc11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:24.886777 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977246c3-f5a5-46c0-8388-af8b9044fc11 None admin] RBAC: Adding query filter params (role.id=9121ba25a6d84756990f0c3a19e9fecb, scope.project.id=933820e2adbc473bbd57a4d59be75d72, user.id=26d24cf0a21d491082b0c2f11e749446) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:24.887090 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977246c3-f5a5-46c0-8388-af8b9044fc11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:24.888489 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977246c3-f5a5-46c0-8388-af8b9044fc11 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:24.894057 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 643/1287] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Sep 9 20:18:24 2022] GET /identity/v3/role_assignments?user.id=26d24cf0a21d491082b0c2f11e749446&scope.project.id=933820e2adbc473bbd57a4d59be75d72&role.id=9121ba25a6d84756990f0c3a19e9fecb => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:18:27.040648 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338be10-c8dd-4b9a-91c3-5e91525c7c8d None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:27.041193 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338be10-c8dd-4b9a-91c3-5e91525c7c8d None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:27.041411 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338be10-c8dd-4b9a-91c3-5e91525c7c8d None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:27.042923 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 645/1288] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:18:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:18:27.049229 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857e55a-e9a8-4fb4-a896-2f89d2628c9b None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:27.049786 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857e55a-e9a8-4fb4-a896-2f89d2628c9b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:27.049938 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857e55a-e9a8-4fb4-a896-2f89d2628c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:27.070932 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e857e55a-e9a8-4fb4-a896-2f89d2628c9b None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:18:27.072269 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e857e55a-e9a8-4fb4-a896-2f89d2628c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:18:27.104845 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:27.142989 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 644/1289] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:18:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:18:27.422434 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5338be10-c8dd-4b9a-91c3-5e91525c7c8d None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:27.522068 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5338be10-c8dd-4b9a-91c3-5e91525c7c8d None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:27.532220 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:27.556353 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8584e5-6eda-45a6-aada-a3ef407687f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:27.556973 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8584e5-6eda-45a6-aada-a3ef407687f9 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:27.557262 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8584e5-6eda-45a6-aada-a3ef407687f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:27.557545 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8584e5-6eda-45a6-aada-a3ef407687f9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:27.558113 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8584e5-6eda-45a6-aada-a3ef407687f9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:27.559466 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8584e5-6eda-45a6-aada-a3ef407687f9 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:27.567990 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-df8584e5-6eda-45a6-aada-a3ef407687f9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 20:18:27.567990 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:18:27.567990 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:18:27.567990 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:18:27.567990 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:18:27.567990 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:18:27.567990 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:18:27.567990 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:18:27.567990 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 20:18:27.567990 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:18:27.567990 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:18:27.567990 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:18:27.567990 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:18:27.567990 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:18:27.567990 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:18:27.567990 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:18:27.567990 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:18:27.573539 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:18:27.573539 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:18:27.573539 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:18:27.573539 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:18:27.573539 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:18:27.573539 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:18:27.573539 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:18:27.573539 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:18:27.573539 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:18:27.573539 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:18:27.573539 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:18:27.573539 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:18:27.573539 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:18:27.573539 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:18:27.573539 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:18:27.573539 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:18:27.574617 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:18:27.574617 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:18:27.574617 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:18:27.574617 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:18:27.574617 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:18:27.574617 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:18:27.574617 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:18:27.574617 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:18:27.574617 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:18:27.574617 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:18:27.574617 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:18:27.574617 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:18:27.574617 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 20:18:27.574617 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:18:27.574617 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 646/1290] 10.222.0.26 () {56 vars in 1117 bytes} [Fri Sep 9 20:18:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:18:27.575662 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857e55a-e9a8-4fb4-a896-2f89d2628c9b None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:27.671416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857e55a-e9a8-4fb4-a896-2f89d2628c9b None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:27.704017 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c292061b-218b-41a0-8edd-c8be17c15b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:27.704565 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292061b-218b-41a0-8edd-c8be17c15b16 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:27.704845 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292061b-218b-41a0-8edd-c8be17c15b16 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:27.705117 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292061b-218b-41a0-8edd-c8be17c15b16 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:27.705773 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c292061b-218b-41a0-8edd-c8be17c15b16 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:27.706074 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c292061b-218b-41a0-8edd-c8be17c15b16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:27.707418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c292061b-218b-41a0-8edd-c8be17c15b16 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:27.712048 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 645/1291] 10.222.0.26 () {56 vars in 1116 bytes} [Fri Sep 9 20:18:27 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:18:27.717748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8584e5-6eda-45a6-aada-a3ef407687f9 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:27.820193 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8584e5-6eda-45a6-aada-a3ef407687f9 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:27.854199 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f030117-e33e-4fc0-b143-d5034df997cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:27.854787 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f030117-e33e-4fc0-b143-d5034df997cd None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:27.855059 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f030117-e33e-4fc0-b143-d5034df997cd None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:27.855326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f030117-e33e-4fc0-b143-d5034df997cd None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:27.855936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f030117-e33e-4fc0-b143-d5034df997cd None admin] RBAC: Adding query filter params (type=placement) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:27.856232 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f030117-e33e-4fc0-b143-d5034df997cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:27.857698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f030117-e33e-4fc0-b143-d5034df997cd None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:27.862973 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 647/1292] 10.222.0.26 () {56 vars in 1116 bytes} [Fri Sep 9 20:18:27 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:18:30.006517 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c362085-e24a-4ef3-96e2-2c311767ac83 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:30.007061 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c362085-e24a-4ef3-96e2-2c311767ac83 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:30.007061 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c362085-e24a-4ef3-96e2-2c311767ac83 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:30.008392 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 646/1293] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:18:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:18:30.013736 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732c6d3-8b0f-4afc-9559-0d016b5244a5 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:30.014112 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732c6d3-8b0f-4afc-9559-0d016b5244a5 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:30.014279 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732c6d3-8b0f-4afc-9559-0d016b5244a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:30.032011 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-d732c6d3-8b0f-4afc-9559-0d016b5244a5 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:18:30.033743 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-d732c6d3-8b0f-4afc-9559-0d016b5244a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:18:30.064249 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:30.103995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 648/1294] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:18:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:18:30.369336 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c362085-e24a-4ef3-96e2-2c311767ac83 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:30.400275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:30.468481 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c362085-e24a-4ef3-96e2-2c311767ac83 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:30.501689 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce9388-58c6-435d-8fa2-56ae6ed303e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:30.502546 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce9388-58c6-435d-8fa2-56ae6ed303e8 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:30.502829 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce9388-58c6-435d-8fa2-56ae6ed303e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:30.503103 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce9388-58c6-435d-8fa2-56ae6ed303e8 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:30.503757 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ce9388-58c6-435d-8fa2-56ae6ed303e8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:30.505103 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ce9388-58c6-435d-8fa2-56ae6ed303e8 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:30.518565 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 647/1295] 10.222.0.26 () {60 vars in 1141 bytes} [Fri Sep 9 20:18:30 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:18:32.675465 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d89c988-99e8-43f5-8b67-1de53a6c0e9c None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:32.675891 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d89c988-99e8-43f5-8b67-1de53a6c0e9c None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:32.676007 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d89c988-99e8-43f5-8b67-1de53a6c0e9c None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:32.677371 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 649/1296] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:18:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:18:32.682451 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f29f61b-9a6c-42ff-ba3e-2f9a4ff6217f None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:32.682840 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f29f61b-9a6c-42ff-ba3e-2f9a4ff6217f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:32.683008 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f29f61b-9a6c-42ff-ba3e-2f9a4ff6217f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:32.701389 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-8f29f61b-9a6c-42ff-ba3e-2f9a4ff6217f None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:18:32.702571 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8f29f61b-9a6c-42ff-ba3e-2f9a4ff6217f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:18:32.772961 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 648/1297] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:18:32 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:18:33.050057 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d89c988-99e8-43f5-8b67-1de53a6c0e9c None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:33.166464 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d89c988-99e8-43f5-8b67-1de53a6c0e9c None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:33.200172 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2925539c-7d55-4e7e-b7be-4beaa1927164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:33.200793 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925539c-7d55-4e7e-b7be-4beaa1927164 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:33.201069 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925539c-7d55-4e7e-b7be-4beaa1927164 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:33.201364 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925539c-7d55-4e7e-b7be-4beaa1927164 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:33.201940 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2925539c-7d55-4e7e-b7be-4beaa1927164 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:33.203291 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2925539c-7d55-4e7e-b7be-4beaa1927164 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:33.211691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-2925539c-7d55-4e7e-b7be-4beaa1927164 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 20:18:33.211691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:18:33.211691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:18:33.211691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:18:33.211691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:18:33.211691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:18:33.211691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:18:33.211691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:18:33.211691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 20:18:33.211691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:18:33.211691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:18:33.211691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:18:33.211691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:18:33.211691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:18:33.211691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:18:33.211691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:18:33.211691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:18:33.212871 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:18:33.212871 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:18:33.212871 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:18:33.212871 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:18:33.212871 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:18:33.212871 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:18:33.212871 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:18:33.212871 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:18:33.212871 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:18:33.212871 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:18:33.212871 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:18:33.212871 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:18:33.212871 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:18:33.212871 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:18:33.212871 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:18:33.212871 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:18:33.214196 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:18:33.214196 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:18:33.214196 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:18:33.214196 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:18:33.214196 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:18:33.214196 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:18:33.214196 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:18:33.214196 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:18:33.214196 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:18:33.214196 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:18:33.214196 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:18:33.214196 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:18:33.214196 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 20:18:33.214196 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:18:33.214196 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 650/1298] 10.222.0.26 () {56 vars in 1117 bytes} [Fri Sep 9 20:18:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:18:33.218770 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f29f61b-9a6c-42ff-ba3e-2f9a4ff6217f None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:33.237028 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:33.316239 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f29f61b-9a6c-42ff-ba3e-2f9a4ff6217f None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:33.348302 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49740928-0909-493c-8c0d-ae3730742b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:33.348896 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49740928-0909-493c-8c0d-ae3730742b58 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:33.349179 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49740928-0909-493c-8c0d-ae3730742b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:33.349480 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49740928-0909-493c-8c0d-ae3730742b58 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:33.350105 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49740928-0909-493c-8c0d-ae3730742b58 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:33.350410 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49740928-0909-493c-8c0d-ae3730742b58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:33.351762 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49740928-0909-493c-8c0d-ae3730742b58 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:33.356652 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 649/1299] 10.222.0.26 () {56 vars in 1116 bytes} [Fri Sep 9 20:18:33 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:18:33.362984 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2925539c-7d55-4e7e-b7be-4beaa1927164 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:33.393945 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:33.459926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2925539c-7d55-4e7e-b7be-4beaa1927164 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:33.496981 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a769bdc3-7834-4ee6-8a8a-2dbbfe8c188a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:33.497611 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769bdc3-7834-4ee6-8a8a-2dbbfe8c188a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:33.497889 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769bdc3-7834-4ee6-8a8a-2dbbfe8c188a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:33.498158 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769bdc3-7834-4ee6-8a8a-2dbbfe8c188a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:33.498850 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a769bdc3-7834-4ee6-8a8a-2dbbfe8c188a None admin] RBAC: Adding query filter params (interface=public, service_id=0c5ba152d7404dd9b8eaad76bd3d7429, region_id=RegionOne) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:33.499154 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a769bdc3-7834-4ee6-8a8a-2dbbfe8c188a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:33.500647 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a769bdc3-7834-4ee6-8a8a-2dbbfe8c188a None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:33.505831 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 651/1300] 10.222.0.26 () {56 vars in 1251 bytes} [Fri Sep 9 20:18:33 2022] GET /identity/v3/endpoints?service_id=0c5ba152d7404dd9b8eaad76bd3d7429&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:18:33.511503 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49740928-0909-493c-8c0d-ae3730742b58 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:33.589977 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:33.608153 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49740928-0909-493c-8c0d-ae3730742b58 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:33.640677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2757bea-6874-42ef-a593-2d99994d6461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:33.641286 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2757bea-6874-42ef-a593-2d99994d6461 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:33.641588 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2757bea-6874-42ef-a593-2d99994d6461 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:33.641891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2757bea-6874-42ef-a593-2d99994d6461 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:33.642487 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2757bea-6874-42ef-a593-2d99994d6461 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:33.643819 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2757bea-6874-42ef-a593-2d99994d6461 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:33.650366 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 650/1301] 10.222.0.26 () {56 vars in 1087 bytes} [Fri Sep 9 20:18:33 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:18:35.762080 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79b7e5-e547-4280-863c-991a55670632 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:35.762605 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79b7e5-e547-4280-863c-991a55670632 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:35.762605 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79b7e5-e547-4280-863c-991a55670632 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:35.763945 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 652/1302] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:18:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:18:35.769810 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc45fc6-a1aa-48e9-ab49-2f9ee27041e5 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:35.770236 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc45fc6-a1aa-48e9-ab49-2f9ee27041e5 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:35.770369 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc45fc6-a1aa-48e9-ab49-2f9ee27041e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:35.788011 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ddc45fc6-a1aa-48e9-ab49-2f9ee27041e5 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:18:35.789180 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ddc45fc6-a1aa-48e9-ab49-2f9ee27041e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:18:35.852711 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 651/1303] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:18:35 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:18:36.123202 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79b7e5-e547-4280-863c-991a55670632 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:36.218754 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:36.226331 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79b7e5-e547-4280-863c-991a55670632 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:36.258474 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69b7555-3bdd-4bf1-9e19-8e37393c2a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:36.259089 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69b7555-3bdd-4bf1-9e19-8e37393c2a9e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:36.259369 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69b7555-3bdd-4bf1-9e19-8e37393c2a9e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:36.259640 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69b7555-3bdd-4bf1-9e19-8e37393c2a9e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:36.260211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69b7555-3bdd-4bf1-9e19-8e37393c2a9e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:36.261587 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69b7555-3bdd-4bf1-9e19-8e37393c2a9e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:36.270521 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-d69b7555-3bdd-4bf1-9e19-8e37393c2a9e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 20:18:36.270521 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:18:36.270521 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:18:36.270521 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:18:36.270521 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:18:36.270521 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:18:36.270521 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:18:36.270521 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:18:36.270521 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 20:18:36.270521 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:18:36.270521 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:18:36.270521 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:18:36.270521 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:18:36.270521 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:18:36.270521 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:18:36.270521 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:18:36.270521 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:18:36.271698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:18:36.271698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:18:36.271698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:18:36.271698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:18:36.271698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:18:36.271698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:18:36.271698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:18:36.271698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:18:36.271698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:18:36.271698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:18:36.271698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:18:36.271698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:18:36.271698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:18:36.271698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:18:36.271698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:18:36.271698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:18:36.272582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:18:36.272582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 09 20:18:36.272582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:18:36.272582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:18:36.272582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:18:36.272582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:18:36.272582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:18:36.272582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:18:36.272582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:18:36.272582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:18:36.272582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:18:36.272582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:18:36.272582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 20:18:36.272582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 09 20:18:36.272582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 653/1304] 10.222.0.26 () {56 vars in 1117 bytes} [Fri Sep 9 20:18:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:18:36.277503 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc45fc6-a1aa-48e9-ab49-2f9ee27041e5 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:36.350822 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:36.373142 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc45fc6-a1aa-48e9-ab49-2f9ee27041e5 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:36.405639 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d618995c-a4e4-4782-877c-a0ce65eea1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:36.406176 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618995c-a4e4-4782-877c-a0ce65eea1f3 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:36.406449 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618995c-a4e4-4782-877c-a0ce65eea1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:36.406718 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618995c-a4e4-4782-877c-a0ce65eea1f3 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:36.407381 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d618995c-a4e4-4782-877c-a0ce65eea1f3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:18:36.407675 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d618995c-a4e4-4782-877c-a0ce65eea1f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:36.409009 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d618995c-a4e4-4782-877c-a0ce65eea1f3 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:36.413883 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 652/1305] 10.222.0.26 () {56 vars in 1116 bytes} [Fri Sep 9 20:18:36 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:18:36.419657 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69b7555-3bdd-4bf1-9e19-8e37393c2a9e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:18:36.516128 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69b7555-3bdd-4bf1-9e19-8e37393c2a9e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:18:36.549497 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9d8ca-73b9-4b94-a973-458b689240fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:18:36.550257 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9d8ca-73b9-4b94-a973-458b689240fb None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:18:36.550537 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9d8ca-73b9-4b94-a973-458b689240fb None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:18:36.551068 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9d8ca-73b9-4b94-a973-458b689240fb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:18:36.551724 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9d8ca-73b9-4b94-a973-458b689240fb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:18:36.553084 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9d8ca-73b9-4b94-a973-458b689240fb None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:18:36.573250 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 654/1306] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:18:36 2022] POST /identity/v3/endpoints => generated 331 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:18:46.411042 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=30 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:18:46.569205 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:19:40.637788 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae94c86-64d8-40af-88e6-168a60df6f1e None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:19:40.638248 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae94c86-64d8-40af-88e6-168a60df6f1e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:19:40.638473 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae94c86-64d8-40af-88e6-168a60df6f1e None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:19:40.641021 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 653/1307] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:19:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:19:40.646342 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9bc270-9c57-4c06-8988-7226c55e22a9 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:19:40.646758 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9bc270-9c57-4c06-8988-7226c55e22a9 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:19:40.646933 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9bc270-9c57-4c06-8988-7226c55e22a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:19:40.748314 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-6e9bc270-9c57-4c06-8988-7226c55e22a9 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:19:40.750127 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6e9bc270-9c57-4c06-8988-7226c55e22a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:19:40.880990 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 655/1308] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:19:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:19:41.185056 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422e139-e119-47ac-a4a2-6fac1fcb48ab None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:19:41.185383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422e139-e119-47ac-a4a2-6fac1fcb48ab None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:19:41.185882 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422e139-e119-47ac-a4a2-6fac1fcb48ab None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:19:41.187285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 654/1309] 10.222.0.26 () {54 vars in 965 bytes} [Fri Sep 9 20:19:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:19:41.193779 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178c582-433b-45fd-af68-7ae485ed3a44 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:19:41.194083 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178c582-433b-45fd-af68-7ae485ed3a44 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:19:41.194331 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178c582-433b-45fd-af68-7ae485ed3a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:19:41.278495 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-a178c582-433b-45fd-af68-7ae485ed3a44 None None] MFA Rules not processed for user `ee5c6e51f2694bf5ae7ea097cf33fb3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:19:41.280262 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a178c582-433b-45fd-af68-7ae485ed3a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:19:41.303960 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:19:41.362454 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 656/1310] 10.222.0.26 () {58 vars in 1062 bytes} [Fri Sep 9 20:19:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:19:41.372102 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5422e139-e119-47ac-a4a2-6fac1fcb48ab None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:19:41.374467 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5422e139-e119-47ac-a4a2-6fac1fcb48ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:19:41.496710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5422e139-e119-47ac-a4a2-6fac1fcb48ab None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:19:41.581689 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b166f-7e6e-4b21-9993-93d9e92fc909 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:19:41.582230 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b166f-7e6e-4b21-9993-93d9e92fc909 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:19:41.582515 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b166f-7e6e-4b21-9993-93d9e92fc909 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:19:41.582793 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b166f-7e6e-4b21-9993-93d9e92fc909 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:19:41.584428 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-874b166f-7e6e-4b21-9993-93d9e92fc909 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:19:41.644665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:19:41.712406 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874b166f-7e6e-4b21-9993-93d9e92fc909 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:19:41.714169 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874b166f-7e6e-4b21-9993-93d9e92fc909 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:19:41.737701 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 655/1311] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:19:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:19:44.182726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27320992-a0bf-4faf-a3ed-c9ba664e6d13 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:19:44.183264 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27320992-a0bf-4faf-a3ed-c9ba664e6d13 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:19:44.183264 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27320992-a0bf-4faf-a3ed-c9ba664e6d13 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:19:44.184538 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 657/1312] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:19:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:19:44.190312 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f216d6a-3bd0-4dcf-af93-cb2fd39c05ab None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:19:44.190738 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f216d6a-3bd0-4dcf-af93-cb2fd39c05ab None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:19:44.190846 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f216d6a-3bd0-4dcf-af93-cb2fd39c05ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:19:44.270263 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-7f216d6a-3bd0-4dcf-af93-cb2fd39c05ab None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:19:44.271533 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7f216d6a-3bd0-4dcf-af93-cb2fd39c05ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:19:44.364875 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 656/1313] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:19:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:19:44.652688 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27320992-a0bf-4faf-a3ed-c9ba664e6d13 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:19:44.654271 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-27320992-a0bf-4faf-a3ed-c9ba664e6d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:19:44.762229 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27320992-a0bf-4faf-a3ed-c9ba664e6d13 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:19:44.835244 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:19:44.837427 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db03a4-2eae-465b-b0ce-1768eb8988b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:19:44.838410 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db03a4-2eae-465b-b0ce-1768eb8988b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:19:44.838846 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db03a4-2eae-465b-b0ce-1768eb8988b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:19:44.839361 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db03a4-2eae-465b-b0ce-1768eb8988b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:19:44.840973 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-c7db03a4-2eae-465b-b0ce-1768eb8988b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:19:44.953452 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7db03a4-2eae-465b-b0ce-1768eb8988b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:19:44.954883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7db03a4-2eae-465b-b0ce-1768eb8988b4 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:19:44.973045 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 658/1314] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:19:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:19:47.421367 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e02c4e-d2f8-49b2-8676-e6a4c97bdb13 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:19:47.421942 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e02c4e-d2f8-49b2-8676-e6a4c97bdb13 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:19:47.421942 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e02c4e-d2f8-49b2-8676-e6a4c97bdb13 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:19:47.423242 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 657/1315] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:19:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:19:47.428370 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda55ae-68ce-462e-81ff-7f41497cb1df None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:19:47.428767 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda55ae-68ce-462e-81ff-7f41497cb1df None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:19:47.428925 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda55ae-68ce-462e-81ff-7f41497cb1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:19:47.493745 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-0eda55ae-68ce-462e-81ff-7f41497cb1df None None] MFA Rules not processed for user `ccfa3b4f784145efae478ef998af8590`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:19:47.494942 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-0eda55ae-68ce-462e-81ff-7f41497cb1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:19:47.545736 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:19:47.590120 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 659/1316] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:19:47 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:19:47.869913 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e02c4e-d2f8-49b2-8676-e6a4c97bdb13 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:19:47.871483 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c8e02c4e-d2f8-49b2-8676-e6a4c97bdb13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:19:47.884451 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:19:47.978941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e02c4e-d2f8-49b2-8676-e6a4c97bdb13 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:19:48.050577 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183dbc7c-5906-48fe-b7c8-a5a5cf5485c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:19:48.051184 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183dbc7c-5906-48fe-b7c8-a5a5cf5485c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:19:48.051465 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183dbc7c-5906-48fe-b7c8-a5a5cf5485c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:19:48.051746 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183dbc7c-5906-48fe-b7c8-a5a5cf5485c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:19:48.053350 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-183dbc7c-5906-48fe-b7c8-a5a5cf5485c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:19:48.113848 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:19:48.159501 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183dbc7c-5906-48fe-b7c8-a5a5cf5485c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:19:48.160904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183dbc7c-5906-48fe-b7c8-a5a5cf5485c2 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:19:48.179399 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 658/1317] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:19:47 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:19:50.871192 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27006d4a-fa26-48c7-95e0-78db8d24f3e1 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:19:50.871723 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27006d4a-fa26-48c7-95e0-78db8d24f3e1 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:19:50.871816 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27006d4a-fa26-48c7-95e0-78db8d24f3e1 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:19:50.873658 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 660/1318] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:19:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:19:50.879263 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb52990-23db-4d8a-b9eb-d5ecce49fa39 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:19:50.879698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb52990-23db-4d8a-b9eb-d5ecce49fa39 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:19:50.879824 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb52990-23db-4d8a-b9eb-d5ecce49fa39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:19:50.944105 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-6bb52990-23db-4d8a-b9eb-d5ecce49fa39 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:19:50.945225 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6bb52990-23db-4d8a-b9eb-d5ecce49fa39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:19:51.039797 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 659/1319] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:19:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:19:51.327461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27006d4a-fa26-48c7-95e0-78db8d24f3e1 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:19:51.329005 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-27006d4a-fa26-48c7-95e0-78db8d24f3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:19:51.430546 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27006d4a-fa26-48c7-95e0-78db8d24f3e1 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:19:51.504600 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a24b3d-d2bb-4204-86de-231d017cf9d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:19:51.505200 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a24b3d-d2bb-4204-86de-231d017cf9d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:19:51.505505 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a24b3d-d2bb-4204-86de-231d017cf9d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:19:51.505774 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a24b3d-d2bb-4204-86de-231d017cf9d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:19:51.507348 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-47a24b3d-d2bb-4204-86de-231d017cf9d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:19:51.514270 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:19:51.616768 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a24b3d-d2bb-4204-86de-231d017cf9d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:19:51.618148 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a24b3d-d2bb-4204-86de-231d017cf9d3 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:19:51.635562 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 661/1320] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:19:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:19:54.437762 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d16368-e561-4567-a592-7f24ef71f208 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:19:54.438338 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d16368-e561-4567-a592-7f24ef71f208 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:19:54.438338 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d16368-e561-4567-a592-7f24ef71f208 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:19:54.439659 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 660/1321] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:19:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:19:54.445077 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777df646-314f-4a54-9a83-813328c842b6 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:19:54.445539 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777df646-314f-4a54-9a83-813328c842b6 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:19:54.445679 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777df646-314f-4a54-9a83-813328c842b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:19:54.520311 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-777df646-314f-4a54-9a83-813328c842b6 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:19:54.521677 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-777df646-314f-4a54-9a83-813328c842b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:19:54.589687 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:19:54.635238 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 662/1322] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:19:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:19:54.917896 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d16368-e561-4567-a592-7f24ef71f208 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:19:54.919382 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f1d16368-e561-4567-a592-7f24ef71f208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:19:54.941989 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:19:55.029549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d16368-e561-4567-a592-7f24ef71f208 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:19:55.114923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4a6f9-5f5b-4ca4-ba34-1b6310d3d770 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:19:55.115540 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4a6f9-5f5b-4ca4-ba34-1b6310d3d770 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:19:55.115823 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4a6f9-5f5b-4ca4-ba34-1b6310d3d770 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:19:55.116105 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4a6f9-5f5b-4ca4-ba34-1b6310d3d770 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:19:55.118568 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0bf4a6f9-5f5b-4ca4-ba34-1b6310d3d770 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:19:55.203590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:19:55.243231 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4a6f9-5f5b-4ca4-ba34-1b6310d3d770 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:19:55.244668 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4a6f9-5f5b-4ca4-ba34-1b6310d3d770 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:19:55.262974 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 661/1323] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:19:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:19:57.459916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9657577-e1f8-4d76-86c6-8188fc881343 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:19:57.460442 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9657577-e1f8-4d76-86c6-8188fc881343 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:19:57.460442 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9657577-e1f8-4d76-86c6-8188fc881343 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:19:57.461963 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 663/1324] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:19:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:19:57.467471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bfc08c-df3f-4700-9cff-a5ada81511e6 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:19:57.467904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bfc08c-df3f-4700-9cff-a5ada81511e6 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:19:57.468095 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bfc08c-df3f-4700-9cff-a5ada81511e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:19:57.533963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f9bfc08c-df3f-4700-9cff-a5ada81511e6 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:19:57.535153 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f9bfc08c-df3f-4700-9cff-a5ada81511e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:19:57.637106 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 662/1325] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:19:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:19:57.914491 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9657577-e1f8-4d76-86c6-8188fc881343 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:19:57.915957 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-b9657577-e1f8-4d76-86c6-8188fc881343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:19:58.016806 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9657577-e1f8-4d76-86c6-8188fc881343 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:19:58.089893 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d59e4b2-3994-49fd-8c14-e346252e6906 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:19:58.090507 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59e4b2-3994-49fd-8c14-e346252e6906 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:19:58.090785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59e4b2-3994-49fd-8c14-e346252e6906 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:19:58.091055 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59e4b2-3994-49fd-8c14-e346252e6906 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:19:58.093207 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-3d59e4b2-3994-49fd-8c14-e346252e6906 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:19:58.110100 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:19:58.202623 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d59e4b2-3994-49fd-8c14-e346252e6906 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:19:58.203980 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d59e4b2-3994-49fd-8c14-e346252e6906 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:19:58.222290 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 664/1326] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:19:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:20:00.427602 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb86a2-f6a0-46fa-89d6-698ddf477b9a None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:00.428156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb86a2-f6a0-46fa-89d6-698ddf477b9a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:00.428156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb86a2-f6a0-46fa-89d6-698ddf477b9a None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:00.429594 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 663/1327] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:20:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:20:00.434851 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae89538-94f7-4a65-b487-29509a79a011 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:00.435758 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae89538-94f7-4a65-b487-29509a79a011 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:00.435936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae89538-94f7-4a65-b487-29509a79a011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:00.502441 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-6ae89538-94f7-4a65-b487-29509a79a011 None None] MFA Rules not processed for user `ccfa3b4f784145efae478ef998af8590`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:20:00.503681 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6ae89538-94f7-4a65-b487-29509a79a011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:00.567672 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:20:00.599845 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 665/1328] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:20:00 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:20:00.880045 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb86a2-f6a0-46fa-89d6-698ddf477b9a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:20:00.881717 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1efb86a2-f6a0-46fa-89d6-698ddf477b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:00.915559 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:20:00.993032 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb86a2-f6a0-46fa-89d6-698ddf477b9a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:20:01.075154 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ccab26-5d6f-4ab9-833e-c849d39612a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:20:01.075819 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ccab26-5d6f-4ab9-833e-c849d39612a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:01.076095 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ccab26-5d6f-4ab9-833e-c849d39612a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:01.076367 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ccab26-5d6f-4ab9-833e-c849d39612a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:01.078015 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-26ccab26-5d6f-4ab9-833e-c849d39612a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:01.155843 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:20:01.189542 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ccab26-5d6f-4ab9-833e-c849d39612a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:20:01.190974 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ccab26-5d6f-4ab9-833e-c849d39612a9 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:20:01.209032 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 664/1329] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:20:00 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:20:03.598259 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92feec-9f10-4cde-b99b-eefcf6ac7df3 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:03.598838 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92feec-9f10-4cde-b99b-eefcf6ac7df3 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:03.598838 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92feec-9f10-4cde-b99b-eefcf6ac7df3 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:03.600148 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 666/1330] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:20:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:20:03.605577 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6888fc-d17a-4a86-b222-8eff8af389c8 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:03.605996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6888fc-d17a-4a86-b222-8eff8af389c8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:03.606108 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6888fc-d17a-4a86-b222-8eff8af389c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:03.671778 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0b6888fc-d17a-4a86-b222-8eff8af389c8 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:20:03.673015 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0b6888fc-d17a-4a86-b222-8eff8af389c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:03.768125 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 665/1331] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:20:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:20:04.057998 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92feec-9f10-4cde-b99b-eefcf6ac7df3 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:20:04.059626 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-fa92feec-9f10-4cde-b99b-eefcf6ac7df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:04.162270 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92feec-9f10-4cde-b99b-eefcf6ac7df3 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:20:04.238902 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bccaa02-60d1-482b-853a-17221df50594 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:20:04.239507 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bccaa02-60d1-482b-853a-17221df50594 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:04.239786 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bccaa02-60d1-482b-853a-17221df50594 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:04.240061 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bccaa02-60d1-482b-853a-17221df50594 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:04.241716 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-3bccaa02-60d1-482b-853a-17221df50594 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:04.268085 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:20:04.348430 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bccaa02-60d1-482b-853a-17221df50594 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:20:04.349895 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bccaa02-60d1-482b-853a-17221df50594 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:20:04.367737 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 667/1332] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:20:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:20:06.565976 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca3372-3c38-497a-aa1d-a83031c9d2d0 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:06.566517 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca3372-3c38-497a-aa1d-a83031c9d2d0 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:06.566517 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca3372-3c38-497a-aa1d-a83031c9d2d0 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:06.567880 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 666/1333] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:20:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:20:06.573215 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0476f1e-05ed-4d18-972b-0f5ee76cda9d None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:06.573686 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0476f1e-05ed-4d18-972b-0f5ee76cda9d None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:06.573784 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0476f1e-05ed-4d18-972b-0f5ee76cda9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:06.639522 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-c0476f1e-05ed-4d18-972b-0f5ee76cda9d None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:20:06.640709 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-c0476f1e-05ed-4d18-972b-0f5ee76cda9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:06.714392 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:20:06.737124 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 668/1334] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:20:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:20:07.031125 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ca3372-3c38-497a-aa1d-a83031c9d2d0 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:20:07.033389 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-10ca3372-3c38-497a-aa1d-a83031c9d2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:07.074894 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:20:07.138841 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ca3372-3c38-497a-aa1d-a83031c9d2d0 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:20:07.212053 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f572b6d0-c4e4-4fd2-8fb4-8ebc42b2bdfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:20:07.212650 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572b6d0-c4e4-4fd2-8fb4-8ebc42b2bdfc service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:07.212925 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572b6d0-c4e4-4fd2-8fb4-8ebc42b2bdfc service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:07.213191 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572b6d0-c4e4-4fd2-8fb4-8ebc42b2bdfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:07.214801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f572b6d0-c4e4-4fd2-8fb4-8ebc42b2bdfc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:07.299741 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:20:07.327442 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f572b6d0-c4e4-4fd2-8fb4-8ebc42b2bdfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:20:07.328890 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f572b6d0-c4e4-4fd2-8fb4-8ebc42b2bdfc service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:20:07.346825 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 667/1335] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:20:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:20:09.824329 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d74c51-8024-457c-bed8-f688594750fd None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:09.824915 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d74c51-8024-457c-bed8-f688594750fd None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:09.824915 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d74c51-8024-457c-bed8-f688594750fd None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:09.826775 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 669/1336] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:20:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:20:09.831895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64355298-39d3-4467-a7af-3fc5b4bbb0d4 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:09.832339 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64355298-39d3-4467-a7af-3fc5b4bbb0d4 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:09.832448 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64355298-39d3-4467-a7af-3fc5b4bbb0d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:09.898007 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-64355298-39d3-4467-a7af-3fc5b4bbb0d4 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:20:09.899220 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-64355298-39d3-4467-a7af-3fc5b4bbb0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:09.994065 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 668/1337] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:20:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:20:10.272374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d74c51-8024-457c-bed8-f688594750fd None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:20:10.273930 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-16d74c51-8024-457c-bed8-f688594750fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:10.376287 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d74c51-8024-457c-bed8-f688594750fd None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:20:10.447243 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a627b523-678c-44ed-972f-7d61528ff0fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:20:10.447847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a627b523-678c-44ed-972f-7d61528ff0fe service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:10.448122 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a627b523-678c-44ed-972f-7d61528ff0fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:10.448389 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a627b523-678c-44ed-972f-7d61528ff0fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:10.450070 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a627b523-678c-44ed-972f-7d61528ff0fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:10.485026 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:20:10.557373 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a627b523-678c-44ed-972f-7d61528ff0fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:20:10.558741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a627b523-678c-44ed-972f-7d61528ff0fe service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:20:10.576721 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 670/1338] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:20:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:20:14.597931 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6843dca-4e07-4236-887d-9f33ad13b4be None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:14.598462 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6843dca-4e07-4236-887d-9f33ad13b4be None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:14.598462 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6843dca-4e07-4236-887d-9f33ad13b4be None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:14.599739 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 669/1339] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:20:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:20:14.604924 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb0652-f29b-4cb0-8a41-3eab696882f4 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:14.605371 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb0652-f29b-4cb0-8a41-3eab696882f4 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:14.605504 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb0652-f29b-4cb0-8a41-3eab696882f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:14.674402 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-0afb0652-f29b-4cb0-8a41-3eab696882f4 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:20:14.675611 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-0afb0652-f29b-4cb0-8a41-3eab696882f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:14.765343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:20:14.776830 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 671/1340] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:20:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:20:15.061051 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6843dca-4e07-4236-887d-9f33ad13b4be None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:20:15.062635 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e6843dca-4e07-4236-887d-9f33ad13b4be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:15.112777 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:20:15.169665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6843dca-4e07-4236-887d-9f33ad13b4be None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:20:15.244806 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ecad22-de7b-4ebd-b54c-da4587d551a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:20:15.245454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecad22-de7b-4ebd-b54c-da4587d551a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:15.245733 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecad22-de7b-4ebd-b54c-da4587d551a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:15.246005 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecad22-de7b-4ebd-b54c-da4587d551a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:15.247640 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-70ecad22-de7b-4ebd-b54c-da4587d551a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:15.345640 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:20:15.362565 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ecad22-de7b-4ebd-b54c-da4587d551a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:20:15.363999 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ecad22-de7b-4ebd-b54c-da4587d551a6 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:20:15.382659 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 670/1341] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:20:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 20:20:18.240758 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f19b2-a053-42f9-b0cd-08db549815c8 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:18.241361 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f19b2-a053-42f9-b0cd-08db549815c8 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:18.241361 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f19b2-a053-42f9-b0cd-08db549815c8 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:18.242696 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 672/1342] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:20:18.248078 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b1100-66de-481c-ae71-410eb5af1c46 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:18.248529 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b1100-66de-481c-ae71-410eb5af1c46 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:18.248658 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b1100-66de-481c-ae71-410eb5af1c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:18.315688 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-8e8b1100-66de-481c-ae71-410eb5af1c46 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:20:18.316894 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8e8b1100-66de-481c-ae71-410eb5af1c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:18.414486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 671/1343] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:20:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:20:18.699339 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f19b2-a053-42f9-b0cd-08db549815c8 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:20:18.701475 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-e16f19b2-a053-42f9-b0cd-08db549815c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:18.810855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f19b2-a053-42f9-b0cd-08db549815c8 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:20:18.885636 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e0614f-1ada-4efd-b882-fef86bbc9dac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:20:18.886245 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0614f-1ada-4efd-b882-fef86bbc9dac service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:18.886524 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0614f-1ada-4efd-b882-fef86bbc9dac service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:18.886789 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0614f-1ada-4efd-b882-fef86bbc9dac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:18.888928 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a2e0614f-1ada-4efd-b882-fef86bbc9dac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:18.932166 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:20:19.005555 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e0614f-1ada-4efd-b882-fef86bbc9dac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:20:19.007011 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e0614f-1ada-4efd-b882-fef86bbc9dac service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:20:19.026346 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 673/1344] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:20:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:20:23.201784 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d36b5f-9f3a-486f-9aac-70e44bc2e5ef None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:23.202259 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d36b5f-9f3a-486f-9aac-70e44bc2e5ef None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:23.202352 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d36b5f-9f3a-486f-9aac-70e44bc2e5ef None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:23.204024 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 672/1345] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:20:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:20:23.209467 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50d998-338b-425e-a217-18ef1362e86a None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:23.209957 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50d998-338b-425e-a217-18ef1362e86a None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:23.210073 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50d998-338b-425e-a217-18ef1362e86a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:23.283059 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-7e50d998-338b-425e-a217-18ef1362e86a None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:20:23.284256 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7e50d998-338b-425e-a217-18ef1362e86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:23.384117 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:20:23.386175 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 674/1346] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:20:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:20:23.669312 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d36b5f-9f3a-486f-9aac-70e44bc2e5ef None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:20:23.670877 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d4d36b5f-9f3a-486f-9aac-70e44bc2e5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:23.729644 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:20:23.784359 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d36b5f-9f3a-486f-9aac-70e44bc2e5ef None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:20:23.862046 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6d860-3b83-47e9-b3c5-18cea7366405 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:20:23.862651 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6d860-3b83-47e9-b3c5-18cea7366405 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:23.862931 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6d860-3b83-47e9-b3c5-18cea7366405 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:23.863204 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6d860-3b83-47e9-b3c5-18cea7366405 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:23.864804 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b7f6d860-3b83-47e9-b3c5-18cea7366405 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:23.975786 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:20:23.980268 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6d860-3b83-47e9-b3c5-18cea7366405 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:20:23.982449 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6d860-3b83-47e9-b3c5-18cea7366405 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:20:24.000638 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 673/1347] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:20:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:20:28.948127 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebbee5-7bb0-47fe-970b-0e052a6caca7 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:28.948127 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebbee5-7bb0-47fe-970b-0e052a6caca7 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:28.948626 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebbee5-7bb0-47fe-970b-0e052a6caca7 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:28.950150 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 675/1348] 10.222.0.26 () {54 vars in 921 bytes} [Fri Sep 9 20:20:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:20:28.955190 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6111947-20f6-4e34-a5e6-8bd81868da1f None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:28.955644 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6111947-20f6-4e34-a5e6-8bd81868da1f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:28.955727 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6111947-20f6-4e34-a5e6-8bd81868da1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:29.054241 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e6111947-20f6-4e34-a5e6-8bd81868da1f None None] MFA Rules not processed for user `26d24cf0a21d491082b0c2f11e749446`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:20:29.055748 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e6111947-20f6-4e34-a5e6-8bd81868da1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:29.225642 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 674/1349] 10.222.0.26 () {58 vars in 1018 bytes} [Fri Sep 9 20:20:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:20:33.377043 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8b405-e52f-4c9d-ad8d-583bf8074c63 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:33.377644 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8b405-e52f-4c9d-ad8d-583bf8074c63 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:33.378024 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8b405-e52f-4c9d-ad8d-583bf8074c63 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:33.379628 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 676/1350] 10.222.0.26 () {54 vars in 921 bytes} [Fri Sep 9 20:20:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:20:33.384852 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092e63c-fe77-497f-a0e3-37f52bb61cef None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:33.385340 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092e63c-fe77-497f-a0e3-37f52bb61cef None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:33.385576 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092e63c-fe77-497f-a0e3-37f52bb61cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:33.444039 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:20:33.469197 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-b092e63c-fe77-497f-a0e3-37f52bb61cef None None] MFA Rules not processed for user `26d24cf0a21d491082b0c2f11e749446`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:20:33.470667 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b092e63c-fe77-497f-a0e3-37f52bb61cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:33.585216 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 675/1351] 10.222.0.26 () {58 vars in 1018 bytes} [Fri Sep 9 20:20:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:20:35.152476 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420ed19-1d74-44f8-ba83-3a779bbad14f None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:35.153006 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420ed19-1d74-44f8-ba83-3a779bbad14f None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:35.153006 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420ed19-1d74-44f8-ba83-3a779bbad14f None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:35.154925 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 677/1352] 10.222.0.26 () {54 vars in 921 bytes} [Fri Sep 9 20:20:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:20:35.159886 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c33ec-cf26-4938-914b-3e3a0dcad01d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:35.160320 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c33ec-cf26-4938-914b-3e3a0dcad01d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:35.160577 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c33ec-cf26-4938-914b-3e3a0dcad01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:35.235226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ec0c33ec-cf26-4938-914b-3e3a0dcad01d None None] MFA Rules not processed for user `26d24cf0a21d491082b0c2f11e749446`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:20:35.236411 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ec0c33ec-cf26-4938-914b-3e3a0dcad01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:35.279627 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:20:35.355410 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 676/1353] 10.222.0.26 () {58 vars in 1018 bytes} [Fri Sep 9 20:20:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:20:38.854443 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d64c10-015c-4371-933a-a15395472ac6 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:38.854950 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d64c10-015c-4371-933a-a15395472ac6 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:38.854950 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d64c10-015c-4371-933a-a15395472ac6 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:38.856387 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 678/1354] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:20:38.862234 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4db301-f9c1-474f-af69-e0fa8e55621f None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:38.862739 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4db301-f9c1-474f-af69-e0fa8e55621f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:38.862838 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4db301-f9c1-474f-af69-e0fa8e55621f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:38.954321 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-5f4db301-f9c1-474f-af69-e0fa8e55621f None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:20:38.955454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5f4db301-f9c1-474f-af69-e0fa8e55621f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:39.058283 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 677/1355] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:20:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:20:39.360873 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e8ae5-88e9-47fa-bb45-377af5d2f384 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:39.361155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e8ae5-88e9-47fa-bb45-377af5d2f384 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:39.361468 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e8ae5-88e9-47fa-bb45-377af5d2f384 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:39.362791 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 679/1356] 10.222.0.26 () {54 vars in 962 bytes} [Fri Sep 9 20:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:20:39.371348 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9623939-df96-42a5-97ad-04b1b035460f None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:39.371881 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9623939-df96-42a5-97ad-04b1b035460f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:39.372956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9623939-df96-42a5-97ad-04b1b035460f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:39.395403 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:20:39.461802 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f9623939-df96-42a5-97ad-04b1b035460f None None] MFA Rules not processed for user `87ece6128f9245f7b49f3e7a9faaed22`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:20:39.463532 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f9623939-df96-42a5-97ad-04b1b035460f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:39.601203 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 678/1357] 10.222.0.26 () {58 vars in 1059 bytes} [Fri Sep 9 20:20:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:20:39.610659 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e8ae5-88e9-47fa-bb45-377af5d2f384 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:20:39.612280 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-9f1e8ae5-88e9-47fa-bb45-377af5d2f384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:39.784627 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e8ae5-88e9-47fa-bb45-377af5d2f384 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:20:39.863324 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:20:39.906768 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df652a27-dd96-4f83-b10d-77599233c903 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:20:39.907385 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df652a27-dd96-4f83-b10d-77599233c903 service nova] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:39.907667 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df652a27-dd96-4f83-b10d-77599233c903 service nova] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:39.907945 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df652a27-dd96-4f83-b10d-77599233c903 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:39.909931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-df652a27-dd96-4f83-b10d-77599233c903 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:40.006696 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df652a27-dd96-4f83-b10d-77599233c903 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:20:40.008305 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df652a27-dd96-4f83-b10d-77599233c903 service nova] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:20:40.030351 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 680/1358] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:20:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:20:42.465740 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50ea4e-0f55-487c-ac0c-b5e18a540f9e None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:42.466821 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50ea4e-0f55-487c-ac0c-b5e18a540f9e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:42.466971 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50ea4e-0f55-487c-ac0c-b5e18a540f9e None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:42.468942 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 679/1359] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:20:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:20:42.474893 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a038aa-8e0d-4c39-9011-d0499b672a14 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:42.475382 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a038aa-8e0d-4c39-9011-d0499b672a14 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:42.475821 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a038aa-8e0d-4c39-9011-d0499b672a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:42.581672 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-79a038aa-8e0d-4c39-9011-d0499b672a14 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:20:42.582936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-79a038aa-8e0d-4c39-9011-d0499b672a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:42.626189 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:20:42.690446 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 681/1360] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:20:42.980416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af50ea4e-0f55-487c-ac0c-b5e18a540f9e None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:20:42.982480 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-af50ea4e-0f55-487c-ac0c-b5e18a540f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:43.056545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:20:43.160223 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af50ea4e-0f55-487c-ac0c-b5e18a540f9e None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:20:43.288661 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ae3a85-ea97-4152-bf41-6802d415078b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:20:43.289307 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae3a85-ea97-4152-bf41-6802d415078b service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:43.289583 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae3a85-ea97-4152-bf41-6802d415078b service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:43.289856 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae3a85-ea97-4152-bf41-6802d415078b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:43.291561 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-39ae3a85-ea97-4152-bf41-6802d415078b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:43.316601 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:20:43.391398 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ae3a85-ea97-4152-bf41-6802d415078b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:20:43.392914 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ae3a85-ea97-4152-bf41-6802d415078b service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:20:43.410899 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 680/1361] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:20:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:20:45.926551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c192118-aeef-44e6-a3da-14b0fbf06e73 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:45.927104 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c192118-aeef-44e6-a3da-14b0fbf06e73 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:45.927104 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c192118-aeef-44e6-a3da-14b0fbf06e73 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:45.928491 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 682/1362] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:20:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:20:45.933676 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08cc05-8671-4058-8f05-df97dd414922 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:45.934095 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08cc05-8671-4058-8f05-df97dd414922 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:45.934311 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08cc05-8671-4058-8f05-df97dd414922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:46.007405 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-3b08cc05-8671-4058-8f05-df97dd414922 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:20:46.008638 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3b08cc05-8671-4058-8f05-df97dd414922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:46.117104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:20:46.129584 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 681/1363] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:20:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:20:46.418612 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54d2e3-1474-45b4-ba95-cce4cdf10ab6 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:46.419063 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54d2e3-1474-45b4-ba95-cce4cdf10ab6 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:46.419442 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54d2e3-1474-45b4-ba95-cce4cdf10ab6 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:46.422183 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 683/1364] 10.222.0.26 () {54 vars in 962 bytes} [Fri Sep 9 20:20:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:20:46.435680 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708be177-2939-4cbe-a1f4-067f78eae4a7 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:46.435965 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708be177-2939-4cbe-a1f4-067f78eae4a7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:46.436237 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708be177-2939-4cbe-a1f4-067f78eae4a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:46.518988 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-708be177-2939-4cbe-a1f4-067f78eae4a7 None None] MFA Rules not processed for user `87ece6128f9245f7b49f3e7a9faaed22`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:20:46.520168 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-708be177-2939-4cbe-a1f4-067f78eae4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:46.647972 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 682/1365] 10.222.0.26 () {58 vars in 1059 bytes} [Fri Sep 9 20:20:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:20:46.658924 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54d2e3-1474-45b4-ba95-cce4cdf10ab6 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:20:46.660690 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-0f54d2e3-1474-45b4-ba95-cce4cdf10ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:46.823015 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54d2e3-1474-45b4-ba95-cce4cdf10ab6 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:20:46.844897 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:20:46.953142 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e63e3-8ab9-4689-9e38-add16eeb1793 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:20:46.953850 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e63e3-8ab9-4689-9e38-add16eeb1793 service nova] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:46.954143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e63e3-8ab9-4689-9e38-add16eeb1793 service nova] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:46.954258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e63e3-8ab9-4689-9e38-add16eeb1793 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:46.955825 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-095e63e3-8ab9-4689-9e38-add16eeb1793 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:47.057098 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e63e3-8ab9-4689-9e38-add16eeb1793 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:20:47.058783 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e63e3-8ab9-4689-9e38-add16eeb1793 service nova] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:20:47.081171 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 684/1366] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:20:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:20:47.174048 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8884a-2d5d-42c7-a133-bd5693f55a9e None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:47.174602 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8884a-2d5d-42c7-a133-bd5693f55a9e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:47.174706 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8884a-2d5d-42c7-a133-bd5693f55a9e None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:47.176555 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 683/1367] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:20:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:20:47.181983 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca250fa-f106-45a1-9138-baa83bbfc71a None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:47.182271 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca250fa-f106-45a1-9138-baa83bbfc71a None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:47.182680 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca250fa-f106-45a1-9138-baa83bbfc71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:47.220903 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:20:47.262399 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-7ca250fa-f106-45a1-9138-baa83bbfc71a None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:20:47.264202 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7ca250fa-f106-45a1-9138-baa83bbfc71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:47.386131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 685/1368] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:20:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:20:47.721821 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a7baa-f866-4a3c-ac4a-8071d5873096 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:47.722207 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a7baa-f866-4a3c-ac4a-8071d5873096 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:47.722551 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a7baa-f866-4a3c-ac4a-8071d5873096 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:47.724123 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 684/1369] 10.222.0.26 () {54 vars in 964 bytes} [Fri Sep 9 20:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:20:47.729024 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf29b7-ac0d-4937-8e4b-0911b0c40172 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:47.729349 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf29b7-ac0d-4937-8e4b-0911b0c40172 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:47.729627 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf29b7-ac0d-4937-8e4b-0911b0c40172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:47.834075 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-8ddf29b7-ac0d-4937-8e4b-0911b0c40172 None None] MFA Rules not processed for user `11d86d2df1cb485592e8a5d7387131ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:20:47.835275 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-8ddf29b7-ac0d-4937-8e4b-0911b0c40172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:47.888688 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:20:47.922845 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 686/1370] 10.222.0.26 () {58 vars in 1061 bytes} [Fri Sep 9 20:20:47 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:20:47.930302 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a7baa-f866-4a3c-ac4a-8071d5873096 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:20:47.932081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-090a7baa-f866-4a3c-ac4a-8071d5873096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:47.987163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:20:48.054164 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a7baa-f866-4a3c-ac4a-8071d5873096 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:20:48.151768 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb2ce6-e251-4720-a283-0e3e772dbfc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:20:48.152387 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb2ce6-e251-4720-a283-0e3e772dbfc7 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:48.152664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb2ce6-e251-4720-a283-0e3e772dbfc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:48.152939 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb2ce6-e251-4720-a283-0e3e772dbfc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:48.155200 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4dfb2ce6-e251-4720-a283-0e3e772dbfc7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:48.407506 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:20:48.433428 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb2ce6-e251-4720-a283-0e3e772dbfc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:20:48.435019 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb2ce6-e251-4720-a283-0e3e772dbfc7 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:20:48.457283 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 685/1371] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:20:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:20:49.337839 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a89f1-968a-4952-8762-b030dfa7f259 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:49.338378 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a89f1-968a-4952-8762-b030dfa7f259 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:49.338378 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a89f1-968a-4952-8762-b030dfa7f259 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:49.339721 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 687/1372] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:20:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:20:49.345572 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec62e6-633f-47c7-9ed9-9a09ce464be4 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:49.345861 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec62e6-633f-47c7-9ed9-9a09ce464be4 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:49.346140 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec62e6-633f-47c7-9ed9-9a09ce464be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:49.412816 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-59ec62e6-633f-47c7-9ed9-9a09ce464be4 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:20:49.413962 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-59ec62e6-633f-47c7-9ed9-9a09ce464be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:49.508677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 686/1373] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:20:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:20:49.796173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543a89f1-968a-4952-8762-b030dfa7f259 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:20:49.797725 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-543a89f1-968a-4952-8762-b030dfa7f259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:49.964605 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543a89f1-968a-4952-8762-b030dfa7f259 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:20:50.032338 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:20:50.101820 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af95782-dbee-4194-bdd8-37fb7a09a335 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:20:50.102433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af95782-dbee-4194-bdd8-37fb7a09a335 service nova] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:50.102717 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af95782-dbee-4194-bdd8-37fb7a09a335 service nova] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:50.102991 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af95782-dbee-4194-bdd8-37fb7a09a335 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:50.104623 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-5af95782-dbee-4194-bdd8-37fb7a09a335 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:50.199725 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af95782-dbee-4194-bdd8-37fb7a09a335 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:20:50.201841 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af95782-dbee-4194-bdd8-37fb7a09a335 service nova] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:20:50.219799 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 688/1374] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:20:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:20:52.483953 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4b256-475d-4352-ba43-d3006d5b887b None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:52.484513 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4b256-475d-4352-ba43-d3006d5b887b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:52.484513 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4b256-475d-4352-ba43-d3006d5b887b None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:52.486094 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 687/1375] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:20:52.491785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b8c39-8b70-49e6-bf8c-0faf4ede1cc4 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:52.492221 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b8c39-8b70-49e6-bf8c-0faf4ede1cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:52.492330 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b8c39-8b70-49e6-bf8c-0faf4ede1cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:52.571093 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-f90b8c39-8b70-49e6-bf8c-0faf4ede1cc4 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:20:52.572404 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-f90b8c39-8b70-49e6-bf8c-0faf4ede1cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:52.580556 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:20:52.682509 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 689/1376] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:20:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:20:52.969027 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4b256-475d-4352-ba43-d3006d5b887b None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:20:52.970715 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0fd4b256-475d-4352-ba43-d3006d5b887b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:53.035182 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:20:53.127398 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4b256-475d-4352-ba43-d3006d5b887b None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:20:53.245898 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c3c7c-b5e2-480e-9568-aa0c7b52879e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:20:53.248533 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c3c7c-b5e2-480e-9568-aa0c7b52879e service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:53.248926 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c3c7c-b5e2-480e-9568-aa0c7b52879e service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:53.249331 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c3c7c-b5e2-480e-9568-aa0c7b52879e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:53.251433 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e04c3c7c-b5e2-480e-9568-aa0c7b52879e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:53.271128 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:20:53.348464 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04c3c7c-b5e2-480e-9568-aa0c7b52879e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:20:53.350264 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04c3c7c-b5e2-480e-9568-aa0c7b52879e service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:20:53.370101 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 688/1377] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:20:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:20:55.598132 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b5f31-cfc7-44f0-90dd-3b4b7dbb7045 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:55.598780 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b5f31-cfc7-44f0-90dd-3b4b7dbb7045 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:55.598780 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b5f31-cfc7-44f0-90dd-3b4b7dbb7045 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:55.600185 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 690/1378] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:20:55.605378 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95b8c2-dcb6-4270-a5ad-4b592f4919c1 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:55.605832 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95b8c2-dcb6-4270-a5ad-4b592f4919c1 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:55.606051 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95b8c2-dcb6-4270-a5ad-4b592f4919c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:55.685832 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-3e95b8c2-dcb6-4270-a5ad-4b592f4919c1 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:20:55.687057 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3e95b8c2-dcb6-4270-a5ad-4b592f4919c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:55.791541 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:20:55.809412 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 689/1379] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:20:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:20:56.105357 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b5f31-cfc7-44f0-90dd-3b4b7dbb7045 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:20:56.107685 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-e96b5f31-cfc7-44f0-90dd-3b4b7dbb7045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:56.262698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:20:56.278476 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b5f31-cfc7-44f0-90dd-3b4b7dbb7045 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:20:56.428830 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd50e55f-7c48-4783-b0fb-0f0f47bf21ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:20:56.429817 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50e55f-7c48-4783-b0fb-0f0f47bf21ed service nova] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:56.430156 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50e55f-7c48-4783-b0fb-0f0f47bf21ed service nova] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:56.430492 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50e55f-7c48-4783-b0fb-0f0f47bf21ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:56.432380 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-dd50e55f-7c48-4783-b0fb-0f0f47bf21ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:56.530196 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd50e55f-7c48-4783-b0fb-0f0f47bf21ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:20:56.532756 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd50e55f-7c48-4783-b0fb-0f0f47bf21ed service nova] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:20:56.547913 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:20:56.558208 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 691/1380] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:20:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:20:59.278302 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29237ccf-716f-4b44-a50b-a5e7efd035ac None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:59.278971 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29237ccf-716f-4b44-a50b-a5e7efd035ac None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:59.278971 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29237ccf-716f-4b44-a50b-a5e7efd035ac None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:59.280168 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 690/1381] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:20:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:20:59.285542 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1fdcc-cc10-49ce-b8c0-a2e4cb76184f None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:20:59.285979 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1fdcc-cc10-49ce-b8c0-a2e4cb76184f None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:20:59.286129 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1fdcc-cc10-49ce-b8c0-a2e4cb76184f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:20:59.352476 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-77a1fdcc-cc10-49ce-b8c0-a2e4cb76184f None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:20:59.353748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-77a1fdcc-cc10-49ce-b8c0-a2e4cb76184f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:59.456424 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 692/1382] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:20:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:20:59.792785 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29237ccf-716f-4b44-a50b-a5e7efd035ac None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:20:59.795083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-29237ccf-716f-4b44-a50b-a5e7efd035ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:20:59.955399 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29237ccf-716f-4b44-a50b-a5e7efd035ac None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:21:00.025444 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:00.072062 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990b81ed-0142-4a5c-9179-dc0e854c7078 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:21:00.072594 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b81ed-0142-4a5c-9179-dc0e854c7078 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:00.072878 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b81ed-0142-4a5c-9179-dc0e854c7078 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:00.073162 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b81ed-0142-4a5c-9179-dc0e854c7078 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:00.075316 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-990b81ed-0142-4a5c-9179-dc0e854c7078 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:00.165645 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990b81ed-0142-4a5c-9179-dc0e854c7078 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:21:00.167234 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990b81ed-0142-4a5c-9179-dc0e854c7078 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:21:00.186511 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 691/1383] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:20:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:21:02.368229 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a38429-5523-418c-bfab-6c3bcd07f289 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:02.368766 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a38429-5523-418c-bfab-6c3bcd07f289 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:02.368852 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a38429-5523-418c-bfab-6c3bcd07f289 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:02.370273 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 693/1384] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:21:02.375855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae8e6da-5044-4551-af74-7abc23c4a36a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:02.376282 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae8e6da-5044-4551-af74-7abc23c4a36a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:02.376388 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae8e6da-5044-4551-af74-7abc23c4a36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:02.448770 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-4ae8e6da-5044-4551-af74-7abc23c4a36a None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:21:02.450677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4ae8e6da-5044-4551-af74-7abc23c4a36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:02.488137 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:02.557837 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 692/1385] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:21:02.897876 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a38429-5523-418c-bfab-6c3bcd07f289 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:21:02.899468 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-b1a38429-5523-418c-bfab-6c3bcd07f289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:02.981449 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:03.054221 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a38429-5523-418c-bfab-6c3bcd07f289 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:21:03.172041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484c249a-5a75-4a31-b4ce-071bff27ecc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:21:03.172705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c249a-5a75-4a31-b4ce-071bff27ecc3 service nova] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:03.173007 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c249a-5a75-4a31-b4ce-071bff27ecc3 service nova] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:03.173308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c249a-5a75-4a31-b4ce-071bff27ecc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:03.175013 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-484c249a-5a75-4a31-b4ce-071bff27ecc3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:03.218998 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:03.275764 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484c249a-5a75-4a31-b4ce-071bff27ecc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:21:03.277699 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484c249a-5a75-4a31-b4ce-071bff27ecc3 service nova] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:21:03.297081 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 694/1386] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:21:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:21:05.559433 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b8c68-d182-4c57-8d4c-9c67edc63b85 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:05.559982 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b8c68-d182-4c57-8d4c-9c67edc63b85 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:05.560074 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b8c68-d182-4c57-8d4c-9c67edc63b85 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:05.561381 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 693/1387] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:21:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:21:05.567899 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45582c2-39f4-44a1-ad1c-bee2913de86f None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:05.568333 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45582c2-39f4-44a1-ad1c-bee2913de86f None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:05.568449 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45582c2-39f4-44a1-ad1c-bee2913de86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:05.642689 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-d45582c2-39f4-44a1-ad1c-bee2913de86f None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:21:05.643896 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-d45582c2-39f4-44a1-ad1c-bee2913de86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:05.742324 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 695/1388] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:21:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:21:06.042557 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7b8c68-d182-4c57-8d4c-9c67edc63b85 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:21:06.044160 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ad7b8c68-d182-4c57-8d4c-9c67edc63b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:06.194366 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7b8c68-d182-4c57-8d4c-9c67edc63b85 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:21:06.212694 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:06.317389 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95fc3a3-e34b-4c7b-9fd5-177500cc5d1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:21:06.318143 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95fc3a3-e34b-4c7b-9fd5-177500cc5d1e service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:06.318486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95fc3a3-e34b-4c7b-9fd5-177500cc5d1e service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:06.318791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95fc3a3-e34b-4c7b-9fd5-177500cc5d1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:06.320680 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f95fc3a3-e34b-4c7b-9fd5-177500cc5d1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:06.420855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95fc3a3-e34b-4c7b-9fd5-177500cc5d1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:21:06.422623 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95fc3a3-e34b-4c7b-9fd5-177500cc5d1e service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:21:06.442971 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 694/1389] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:21:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:21:08.654749 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd42ed-ade6-4cfb-92e0-0e0c95a9a13a None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:08.655316 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd42ed-ade6-4cfb-92e0-0e0c95a9a13a None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:08.655316 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd42ed-ade6-4cfb-92e0-0e0c95a9a13a None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:08.656796 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 696/1390] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:21:08.662313 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edae1be-2be9-4644-8b7b-29ac040c3d12 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:08.662799 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edae1be-2be9-4644-8b7b-29ac040c3d12 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:08.662931 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edae1be-2be9-4644-8b7b-29ac040c3d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:08.692349 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:08.734496 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-9edae1be-2be9-4644-8b7b-29ac040c3d12 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:21:08.735703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9edae1be-2be9-4644-8b7b-29ac040c3d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:08.832584 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 695/1391] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:21:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:21:09.125405 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd42ed-ade6-4cfb-92e0-0e0c95a9a13a None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:21:09.126910 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-43fd42ed-ade6-4cfb-92e0-0e0c95a9a13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:09.140721 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:09.281547 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd42ed-ade6-4cfb-92e0-0e0c95a9a13a None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:21:09.368718 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:09.397413 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321b40c-fce1-4a89-a449-1a275e756a81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:21:09.398016 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321b40c-fce1-4a89-a449-1a275e756a81 service nova] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:09.398297 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321b40c-fce1-4a89-a449-1a275e756a81 service nova] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:09.398569 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321b40c-fce1-4a89-a449-1a275e756a81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:09.400157 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6321b40c-fce1-4a89-a449-1a275e756a81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:09.492610 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6321b40c-fce1-4a89-a449-1a275e756a81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:21:09.494277 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6321b40c-fce1-4a89-a449-1a275e756a81 service nova] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:21:09.516917 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 697/1392] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:21:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:21:09.615166 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106b540-55e3-473d-a494-f3251283abfe None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:09.615500 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106b540-55e3-473d-a494-f3251283abfe None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:09.615801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106b540-55e3-473d-a494-f3251283abfe None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:09.617185 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 696/1393] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:21:09.623417 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d441fb-6ca4-47ef-9eec-612a421f4a40 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:09.623669 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d441fb-6ca4-47ef-9eec-612a421f4a40 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:09.623943 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d441fb-6ca4-47ef-9eec-612a421f4a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:09.699280 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-a0d441fb-6ca4-47ef-9eec-612a421f4a40 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:21:09.700510 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a0d441fb-6ca4-47ef-9eec-612a421f4a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:09.755272 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:09.803070 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 698/1394] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:21:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:21:10.103075 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d104ec8-7e72-4e8e-9053-6792caf81963 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:10.103534 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d104ec8-7e72-4e8e-9053-6792caf81963 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:10.103654 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d104ec8-7e72-4e8e-9053-6792caf81963 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:10.104959 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 697/1395] 10.222.0.26 () {54 vars in 966 bytes} [Fri Sep 9 20:21:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:21:10.110090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8455fb-0cad-478a-a823-05cab143c390 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:10.110527 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8455fb-0cad-478a-a823-05cab143c390 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:10.110797 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8455fb-0cad-478a-a823-05cab143c390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:10.185679 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-9b8455fb-0cad-478a-a823-05cab143c390 None None] MFA Rules not processed for user `5298486310004b63b51116dbc747e31d`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:21:10.186780 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-9b8455fb-0cad-478a-a823-05cab143c390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:10.263456 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 699/1396] 10.222.0.26 () {58 vars in 1063 bytes} [Fri Sep 9 20:21:10 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:21:10.272270 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d104ec8-7e72-4e8e-9053-6792caf81963 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:21:10.273878 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6d104ec8-7e72-4e8e-9053-6792caf81963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:10.376966 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:10.379258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d104ec8-7e72-4e8e-9053-6792caf81963 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:21:10.456311 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c3048f-6984-404d-b31d-75ba537276c9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5298486310004b63b51116dbc747e31d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:21:10.456853 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c3048f-6984-404d-b31d-75ba537276c9 service glance] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:10.457134 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c3048f-6984-404d-b31d-75ba537276c9 service glance] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:10.457441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c3048f-6984-404d-b31d-75ba537276c9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:10.459052 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b4c3048f-6984-404d-b31d-75ba537276c9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:10.571184 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c3048f-6984-404d-b31d-75ba537276c9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:21:10.572610 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c3048f-6984-404d-b31d-75ba537276c9 service glance] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:21:10.590509 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 698/1397] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:21:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:21:11.345339 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09147e1-6b5d-4154-b9fc-3797671cd25e None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:11.345863 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09147e1-6b5d-4154-b9fc-3797671cd25e None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:11.345978 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09147e1-6b5d-4154-b9fc-3797671cd25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:11.405862 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:11.415239 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-f09147e1-6b5d-4154-b9fc-3797671cd25e None None] MFA Rules not processed for user `cedaf156c5584d62beedde598c94f322`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:21:11.416414 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-f09147e1-6b5d-4154-b9fc-3797671cd25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:11.507562 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 700/1398] 10.222.0.26 () {58 vars in 1009 bytes} [Fri Sep 9 20:21:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:21:11.552931 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac73a1-5a86-46b1-a6b8-f510b2cc6ff7 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:11.553463 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac73a1-5a86-46b1-a6b8-f510b2cc6ff7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:11.553573 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac73a1-5a86-46b1-a6b8-f510b2cc6ff7 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:11.554839 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 699/1399] 10.222.0.26 () {54 vars in 943 bytes} [Fri Sep 9 20:21:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:21:11.561908 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b1d7ed-f5f0-46c7-a1dd-137d36caf708 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:11.562195 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b1d7ed-f5f0-46c7-a1dd-137d36caf708 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:11.562470 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b1d7ed-f5f0-46c7-a1dd-137d36caf708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:11.644359 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-a0b1d7ed-f5f0-46c7-a1dd-137d36caf708 None None] MFA Rules not processed for user `fcb62d06f7e94e9295c54504ddabb1d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:21:11.645672 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a0b1d7ed-f5f0-46c7-a1dd-137d36caf708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:11.720989 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:11.744437 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e5904-9186-4982-965d-7931f43bc60d None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:11.744742 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e5904-9186-4982-965d-7931f43bc60d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:11.745015 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e5904-9186-4982-965d-7931f43bc60d None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:11.746904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 700/1400] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:21:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:21:11.752437 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bb844-6f64-499a-a3a6-3bad75a96e20 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:11.752733 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bb844-6f64-499a-a3a6-3bad75a96e20 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:11.753013 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bb844-6f64-499a-a3a6-3bad75a96e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:11.758780 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 701/1401] 10.222.0.26 () {58 vars in 1040 bytes} [Fri Sep 9 20:21:11 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:21:11.769429 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b1d7ed-f5f0-46c7-a1dd-137d36caf708 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:21:11.778632 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b1d7ed-f5f0-46c7-a1dd-137d36caf708 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:21:11.789928 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:11.820208 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f57bb844-6f64-499a-a3a6-3bad75a96e20 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:21:11.821453 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f57bb844-6f64-499a-a3a6-3bad75a96e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:11.884221 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f84e2b1-d5df-41c4-af86-c150597b6677 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb62d06f7e94e9295c54504ddabb1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:21:11.884825 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f84e2b1-d5df-41c4-af86-c150597b6677 service swift] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:11.885117 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f84e2b1-d5df-41c4-af86-c150597b6677 service swift] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:11.885422 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f84e2b1-d5df-41c4-af86-c150597b6677 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:11.887612 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-4f84e2b1-d5df-41c4-af86-c150597b6677 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:11.916600 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 701/1402] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:21:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:21:11.968231 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:11.971119 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f84e2b1-d5df-41c4-af86-c150597b6677 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:21:11.973015 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f84e2b1-d5df-41c4-af86-c150597b6677 service swift] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:21:11.981372 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 702/1403] 10.222.0.26 () {60 vars in 1384 bytes} [Fri Sep 9 20:21:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 20:21:12.206206 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57bb844-6f64-499a-a3a6-3bad75a96e20 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:21:12.207615 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f57bb844-6f64-499a-a3a6-3bad75a96e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:12.332111 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:12.336434 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57bb844-6f64-499a-a3a6-3bad75a96e20 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:21:12.477229 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064b4b5-828f-4526-8765-5c7e387e0f5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:21:12.477890 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064b4b5-828f-4526-8765-5c7e387e0f5e service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:12.478169 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064b4b5-828f-4526-8765-5c7e387e0f5e service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:12.478441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064b4b5-828f-4526-8765-5c7e387e0f5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:12.480163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7064b4b5-828f-4526-8765-5c7e387e0f5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:12.579526 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7064b4b5-828f-4526-8765-5c7e387e0f5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:21:12.581119 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7064b4b5-828f-4526-8765-5c7e387e0f5e service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:21:12.601665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 702/1404] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:21:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:21:14.821814 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311e1bc-3701-4eb6-8236-970a1b1d0c2f None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:14.822431 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311e1bc-3701-4eb6-8236-970a1b1d0c2f None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:14.822431 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311e1bc-3701-4eb6-8236-970a1b1d0c2f None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:14.824416 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 703/1405] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:21:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:21:14.830076 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bea4ee-baa9-41e3-a5f3-101a2a4e61d0 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:14.831011 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bea4ee-baa9-41e3-a5f3-101a2a4e61d0 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:14.831295 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bea4ee-baa9-41e3-a5f3-101a2a4e61d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:14.835887 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:14.909063 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-11bea4ee-baa9-41e3-a5f3-101a2a4e61d0 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:21:14.910391 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-11bea4ee-baa9-41e3-a5f3-101a2a4e61d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:15.021655 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 703/1406] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:21:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:21:15.304930 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e311e1bc-3701-4eb6-8236-970a1b1d0c2f None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:21:15.306499 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-e311e1bc-3701-4eb6-8236-970a1b1d0c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:15.472546 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e311e1bc-3701-4eb6-8236-970a1b1d0c2f None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:21:15.552676 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:15.606064 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc376d80-29f4-4029-9b8f-655c60453983 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:21:15.606824 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc376d80-29f4-4029-9b8f-655c60453983 service nova] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:15.607198 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc376d80-29f4-4029-9b8f-655c60453983 service nova] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:15.607577 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc376d80-29f4-4029-9b8f-655c60453983 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:15.609267 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-cc376d80-29f4-4029-9b8f-655c60453983 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:15.711731 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc376d80-29f4-4029-9b8f-655c60453983 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:21:15.713487 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc376d80-29f4-4029-9b8f-655c60453983 service nova] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:21:15.731967 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 704/1407] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:21:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:21:25.011255 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=64 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:25.730021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=64 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:34.291133 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64c1bc-e72b-4112-8725-cd5652ae7f6f None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:34.291681 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64c1bc-e72b-4112-8725-cd5652ae7f6f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:34.291778 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64c1bc-e72b-4112-8725-cd5652ae7f6f None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:34.293055 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 704/1408] 10.222.0.26 () {54 vars in 918 bytes} [Fri Sep 9 20:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:21:34.298281 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f98206-896e-42ef-9f50-7bcd8e848849 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:34.298722 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f98206-896e-42ef-9f50-7bcd8e848849 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:34.298832 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f98206-896e-42ef-9f50-7bcd8e848849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:34.373886 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-e0f98206-896e-42ef-9f50-7bcd8e848849 None None] MFA Rules not processed for user `26d24cf0a21d491082b0c2f11e749446`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:21:34.375331 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-e0f98206-896e-42ef-9f50-7bcd8e848849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:34.480456 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 705/1409] 10.222.0.26 () {58 vars in 1015 bytes} [Fri Sep 9 20:21:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:21:34.494599 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2bfca4-8894-4f35-9e41-eefa123f6b5b None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:34.494895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2bfca4-8894-4f35-9e41-eefa123f6b5b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:34.495173 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2bfca4-8894-4f35-9e41-eefa123f6b5b None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:34.496605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 705/1410] 10.222.0.26 () {54 vars in 918 bytes} [Fri Sep 9 20:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:21:34.501309 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd92ca-89d0-4786-9657-464fefddc140 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:34.501719 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd92ca-89d0-4786-9657-464fefddc140 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:34.501998 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd92ca-89d0-4786-9657-464fefddc140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:34.572728 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:34.576484 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-24bd92ca-89d0-4786-9657-464fefddc140 None None] MFA Rules not processed for user `87ece6128f9245f7b49f3e7a9faaed22`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:21:34.578275 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-24bd92ca-89d0-4786-9657-464fefddc140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:34.691841 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 706/1411] 10.222.0.26 () {58 vars in 1015 bytes} [Fri Sep 9 20:21:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:21:37.233817 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b01f67-f7aa-4582-89c1-81a235bca07f None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:37.234385 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b01f67-f7aa-4582-89c1-81a235bca07f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:37.234385 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b01f67-f7aa-4582-89c1-81a235bca07f None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:37.235709 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 706/1412] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 20:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:21:37.240810 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa4819-d606-436d-ae2c-91e5615d562f None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:37.241204 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa4819-d606-436d-ae2c-91e5615d562f None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:37.241384 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa4819-d606-436d-ae2c-91e5615d562f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:37.310960 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-72aa4819-d606-436d-ae2c-91e5615d562f None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:21:37.312427 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-72aa4819-d606-436d-ae2c-91e5615d562f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:37.367506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:37.409577 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 707/1413] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 20:21:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:21:37.701271 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b01f67-f7aa-4582-89c1-81a235bca07f None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:21:37.702847 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-33b01f67-f7aa-4582-89c1-81a235bca07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:37.846700 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b01f67-f7aa-4582-89c1-81a235bca07f None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:21:37.919793 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:37.959232 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e797df-c52e-48e1-9e11-61c124cb932a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:21:37.959834 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e797df-c52e-48e1-9e11-61c124cb932a service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:37.960115 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e797df-c52e-48e1-9e11-61c124cb932a service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:37.960383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e797df-c52e-48e1-9e11-61c124cb932a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:37.961993 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f9e797df-c52e-48e1-9e11-61c124cb932a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:38.051242 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e797df-c52e-48e1-9e11-61c124cb932a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:21:38.052818 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e797df-c52e-48e1-9e11-61c124cb932a service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:21:38.071742 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 707/1414] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:21:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:21:39.201029 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a703e-2c36-4665-872c-e8e121bfcce1 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:39.201613 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a703e-2c36-4665-872c-e8e121bfcce1 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:39.201613 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a703e-2c36-4665-872c-e8e121bfcce1 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:39.202932 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 708/1415] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 20:21:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:21:39.208878 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e68773-5c50-42a8-ac85-4402595a63d2 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:39.209378 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e68773-5c50-42a8-ac85-4402595a63d2 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:39.209491 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e68773-5c50-42a8-ac85-4402595a63d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:39.292686 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-04e68773-5c50-42a8-ac85-4402595a63d2 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:21:39.293963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-04e68773-5c50-42a8-ac85-4402595a63d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:39.324770 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:39.394143 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 708/1416] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 20:21:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:21:39.545645 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a703e-2c36-4665-872c-e8e121bfcce1 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:21:39.547150 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-389a703e-2c36-4665-872c-e8e121bfcce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:39.699485 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a703e-2c36-4665-872c-e8e121bfcce1 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:21:39.738544 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:39.811705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc3ea8-a106-4e52-8bde-1f4e3f1e59f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:21:39.812213 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc3ea8-a106-4e52-8bde-1f4e3f1e59f8 service nova] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:39.812487 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc3ea8-a106-4e52-8bde-1f4e3f1e59f8 service nova] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:39.812753 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc3ea8-a106-4e52-8bde-1f4e3f1e59f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:39.814426 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-c8bc3ea8-a106-4e52-8bde-1f4e3f1e59f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:39.905531 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc3ea8-a106-4e52-8bde-1f4e3f1e59f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:21:39.907144 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc3ea8-a106-4e52-8bde-1f4e3f1e59f8 service nova] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:21:39.925019 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 709/1417] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:21:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:21:41.045727 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79be1e-f9b5-44bb-860b-97b956bd7188 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:41.046274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79be1e-f9b5-44bb-860b-97b956bd7188 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:41.046372 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79be1e-f9b5-44bb-860b-97b956bd7188 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:41.047640 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 709/1418] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 20:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:21:41.053921 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5af213-254a-4aea-96c7-d46c606e18d0 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:41.054375 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5af213-254a-4aea-96c7-d46c606e18d0 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:41.054480 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5af213-254a-4aea-96c7-d46c606e18d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:41.108421 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:41.130221 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-0b5af213-254a-4aea-96c7-d46c606e18d0 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:21:41.131444 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-0b5af213-254a-4aea-96c7-d46c606e18d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:41.225745 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 710/1419] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 20:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:21:41.378922 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79be1e-f9b5-44bb-860b-97b956bd7188 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:21:41.380351 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5c79be1e-f9b5-44bb-860b-97b956bd7188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:41.522916 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79be1e-f9b5-44bb-860b-97b956bd7188 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:21:41.534747 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:41.636445 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f43f02-20b6-4b35-b22c-1bd8d494b355 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:21:41.636967 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f43f02-20b6-4b35-b22c-1bd8d494b355 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:41.637269 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f43f02-20b6-4b35-b22c-1bd8d494b355 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:41.637549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f43f02-20b6-4b35-b22c-1bd8d494b355 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:41.639156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b1f43f02-20b6-4b35-b22c-1bd8d494b355 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:41.731808 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f43f02-20b6-4b35-b22c-1bd8d494b355 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:21:41.733437 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f43f02-20b6-4b35-b22c-1bd8d494b355 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:21:41.751194 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 710/1420] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:21:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:21:42.859304 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e116d05-b196-47c9-a1a5-3a96431ff298 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:42.859881 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e116d05-b196-47c9-a1a5-3a96431ff298 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:42.859881 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e116d05-b196-47c9-a1a5-3a96431ff298 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:42.861194 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 711/1421] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 20:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:21:42.867271 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c1ec9-3bbf-4b64-a679-6bda3b1cf7ad None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:42.867712 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c1ec9-3bbf-4b64-a679-6bda3b1cf7ad None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:42.867842 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c1ec9-3bbf-4b64-a679-6bda3b1cf7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:42.888231 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:42.943516 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0b1c1ec9-3bbf-4b64-a679-6bda3b1cf7ad None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:21:42.944659 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0b1c1ec9-3bbf-4b64-a679-6bda3b1cf7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:43.042881 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 711/1422] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 20:21:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:21:43.188962 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e116d05-b196-47c9-a1a5-3a96431ff298 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:21:43.190516 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-4e116d05-b196-47c9-a1a5-3a96431ff298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:43.300045 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:43.334915 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e116d05-b196-47c9-a1a5-3a96431ff298 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:21:43.445425 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf20e1-ecc7-414a-821a-46ae950d0e53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:21:43.446038 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf20e1-ecc7-414a-821a-46ae950d0e53 service nova] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:43.446314 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf20e1-ecc7-414a-821a-46ae950d0e53 service nova] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:43.446586 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf20e1-ecc7-414a-821a-46ae950d0e53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:43.448196 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-f4cf20e1-ecc7-414a-821a-46ae950d0e53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:43.527965 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:43.541299 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cf20e1-ecc7-414a-821a-46ae950d0e53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:21:43.542924 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cf20e1-ecc7-414a-821a-46ae950d0e53 service nova] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:21:43.560195 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 712/1423] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:21:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:21:44.661617 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b65ba3-e7c4-41c3-88d8-595cdae57cd7 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:44.662175 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b65ba3-e7c4-41c3-88d8-595cdae57cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:44.662175 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b65ba3-e7c4-41c3-88d8-595cdae57cd7 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:44.663497 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 712/1424] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 20:21:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:21:44.670324 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb84cf-f91a-45f2-9de2-8d6a86617d62 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:44.670795 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb84cf-f91a-45f2-9de2-8d6a86617d62 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:44.670910 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb84cf-f91a-45f2-9de2-8d6a86617d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:44.744761 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-eebb84cf-f91a-45f2-9de2-8d6a86617d62 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:21:44.745990 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-eebb84cf-f91a-45f2-9de2-8d6a86617d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:44.846717 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 713/1425] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 20:21:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:21:44.997180 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b65ba3-e7c4-41c3-88d8-595cdae57cd7 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:21:44.998748 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a4b65ba3-e7c4-41c3-88d8-595cdae57cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:45.082347 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:45.147129 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b65ba3-e7c4-41c3-88d8-595cdae57cd7 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:21:45.256386 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4836aa58-a90e-46d8-804d-1e63c7fcf9db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:21:45.256990 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836aa58-a90e-46d8-804d-1e63c7fcf9db service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:45.257288 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836aa58-a90e-46d8-804d-1e63c7fcf9db service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:45.257563 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836aa58-a90e-46d8-804d-1e63c7fcf9db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:45.259152 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4836aa58-a90e-46d8-804d-1e63c7fcf9db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:45.307342 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:45.348754 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4836aa58-a90e-46d8-804d-1e63c7fcf9db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:21:45.350352 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4836aa58-a90e-46d8-804d-1e63c7fcf9db service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:21:45.367752 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 713/1426] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:21:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:21:46.413841 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452bcfd-73ca-41ce-81cf-6d65a4bf0453 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:46.414420 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452bcfd-73ca-41ce-81cf-6d65a4bf0453 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:46.414420 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452bcfd-73ca-41ce-81cf-6d65a4bf0453 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:46.415798 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 714/1427] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 20:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:21:46.422509 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d7398-006c-4deb-bf39-7584c579366f None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:46.422949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d7398-006c-4deb-bf39-7584c579366f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:46.423046 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d7398-006c-4deb-bf39-7584c579366f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:46.497108 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-797d7398-006c-4deb-bf39-7584c579366f None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:21:46.498349 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-797d7398-006c-4deb-bf39-7584c579366f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:46.600928 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 714/1428] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 20:21:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:21:46.750597 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452bcfd-73ca-41ce-81cf-6d65a4bf0453 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:21:46.752145 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-1452bcfd-73ca-41ce-81cf-6d65a4bf0453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:46.803692 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:46.897904 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452bcfd-73ca-41ce-81cf-6d65a4bf0453 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:21:47.006783 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad27ea-90f2-455e-9df9-a842874c23a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:21:47.007378 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad27ea-90f2-455e-9df9-a842874c23a6 service nova] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:47.007648 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad27ea-90f2-455e-9df9-a842874c23a6 service nova] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:47.007910 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad27ea-90f2-455e-9df9-a842874c23a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:47.009507 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-edad27ea-90f2-455e-9df9-a842874c23a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:47.016168 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:47.097814 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad27ea-90f2-455e-9df9-a842874c23a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:21:47.099393 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad27ea-90f2-455e-9df9-a842874c23a6 service nova] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:21:47.117099 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 715/1429] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:21:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:21:48.161998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f6095-8588-4876-97bf-f4df8a2d3424 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:48.162568 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f6095-8588-4876-97bf-f4df8a2d3424 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:48.162568 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f6095-8588-4876-97bf-f4df8a2d3424 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:48.163881 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 715/1430] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 20:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:21:48.170066 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b053291-aa88-4e10-a491-ca3ae304649f None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:48.170506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b053291-aa88-4e10-a491-ca3ae304649f None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:48.170624 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b053291-aa88-4e10-a491-ca3ae304649f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:48.243926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-4b053291-aa88-4e10-a491-ca3ae304649f None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:21:48.245134 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-4b053291-aa88-4e10-a491-ca3ae304649f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:48.311002 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:48.343141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 716/1431] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 20:21:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:21:48.492451 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f6095-8588-4876-97bf-f4df8a2d3424 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:21:48.494054 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-259f6095-8588-4876-97bf-f4df8a2d3424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:48.512486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:48.638689 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f6095-8588-4876-97bf-f4df8a2d3424 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:21:48.726880 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:48.753446 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54611152-62b7-4ec0-853f-022dbe349fd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:21:48.753990 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54611152-62b7-4ec0-853f-022dbe349fd4 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:48.754264 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54611152-62b7-4ec0-853f-022dbe349fd4 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:48.754531 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54611152-62b7-4ec0-853f-022dbe349fd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:48.756104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-54611152-62b7-4ec0-853f-022dbe349fd4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:48.843636 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54611152-62b7-4ec0-853f-022dbe349fd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:21:48.845253 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54611152-62b7-4ec0-853f-022dbe349fd4 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:21:48.863394 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 716/1432] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:21:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:21:49.904379 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2362b7-23ac-44d7-9146-ae131ff4e6cb None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:49.904941 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2362b7-23ac-44d7-9146-ae131ff4e6cb None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:49.904941 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2362b7-23ac-44d7-9146-ae131ff4e6cb None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:49.906294 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 717/1433] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 20:21:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:21:49.912486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5cb762-1144-425e-9424-e271583c0bd8 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:49.912927 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5cb762-1144-425e-9424-e271583c0bd8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:49.913026 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5cb762-1144-425e-9424-e271583c0bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:49.988281 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-6e5cb762-1144-425e-9424-e271583c0bd8 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:21:49.989474 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6e5cb762-1144-425e-9424-e271583c0bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:50.029564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:50.083112 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 717/1434] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 20:21:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:21:50.231510 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2362b7-23ac-44d7-9146-ae131ff4e6cb None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:21:50.233070 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-4d2362b7-23ac-44d7-9146-ae131ff4e6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:50.383697 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2362b7-23ac-44d7-9146-ae131ff4e6cb None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:21:50.439065 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:50.497392 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3ccd8-a8b3-46d2-90da-33bc653308a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:21:50.497937 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3ccd8-a8b3-46d2-90da-33bc653308a3 service nova] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:50.498218 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3ccd8-a8b3-46d2-90da-33bc653308a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:50.498487 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3ccd8-a8b3-46d2-90da-33bc653308a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:50.500333 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-fab3ccd8-a8b3-46d2-90da-33bc653308a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:50.587658 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3ccd8-a8b3-46d2-90da-33bc653308a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:21:50.589848 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3ccd8-a8b3-46d2-90da-33bc653308a3 service nova] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:21:50.607267 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 718/1435] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:21:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:21:51.672718 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d15944b-65f4-4dca-a319-ff7b97917d0b None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:51.673129 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d15944b-65f4-4dca-a319-ff7b97917d0b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:51.673419 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d15944b-65f4-4dca-a319-ff7b97917d0b None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:51.674799 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 718/1436] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 20:21:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:21:51.681037 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf28340-ca7a-4d78-96c3-a15f935df63f None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:51.681510 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf28340-ca7a-4d78-96c3-a15f935df63f None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:51.681624 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf28340-ca7a-4d78-96c3-a15f935df63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:51.759180 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-bdf28340-ca7a-4d78-96c3-a15f935df63f None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:21:51.760867 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-bdf28340-ca7a-4d78-96c3-a15f935df63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:51.762883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:51.856778 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 719/1437] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 20:21:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:21:52.006951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d15944b-65f4-4dca-a319-ff7b97917d0b None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:21:52.008524 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0d15944b-65f4-4dca-a319-ff7b97917d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:52.152347 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d15944b-65f4-4dca-a319-ff7b97917d0b None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:21:52.178651 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:52.263261 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e105f-ac55-4410-8d0c-6e151af24d94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:21:52.264071 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e105f-ac55-4410-8d0c-6e151af24d94 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:52.264356 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e105f-ac55-4410-8d0c-6e151af24d94 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:52.264629 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e105f-ac55-4410-8d0c-6e151af24d94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:52.266251 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-259e105f-ac55-4410-8d0c-6e151af24d94 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:52.355212 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e105f-ac55-4410-8d0c-6e151af24d94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:21:52.356801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e105f-ac55-4410-8d0c-6e151af24d94 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:21:52.375414 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 719/1438] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:21:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:21:54.468202 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ee482-a332-4938-81e0-aa05f8728f95 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:54.468778 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ee482-a332-4938-81e0-aa05f8728f95 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:54.468778 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ee482-a332-4938-81e0-aa05f8728f95 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:54.470100 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 720/1439] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 20:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:21:54.475448 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0e95a-b5ee-4ceb-ab33-e4f46b71463a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:54.475839 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0e95a-b5ee-4ceb-ab33-e4f46b71463a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:54.476011 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0e95a-b5ee-4ceb-ab33-e4f46b71463a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:54.516539 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:54.541792 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-dfd0e95a-b5ee-4ceb-ab33-e4f46b71463a None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:21:54.543007 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-dfd0e95a-b5ee-4ceb-ab33-e4f46b71463a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:54.646484 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 720/1440] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 20:21:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:21:54.936260 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ee482-a332-4938-81e0-aa05f8728f95 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:21:54.937840 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-3b7ee482-a332-4938-81e0-aa05f8728f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:55.046679 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ee482-a332-4938-81e0-aa05f8728f95 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:21:55.079916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:55.120953 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc2338-9251-40a2-aba7-9e13eba6ab61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:21:55.121608 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc2338-9251-40a2-aba7-9e13eba6ab61 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:55.121919 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc2338-9251-40a2-aba7-9e13eba6ab61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:55.122191 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc2338-9251-40a2-aba7-9e13eba6ab61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:55.124109 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-51cc2338-9251-40a2-aba7-9e13eba6ab61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:55.230152 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc2338-9251-40a2-aba7-9e13eba6ab61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:21:55.231586 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc2338-9251-40a2-aba7-9e13eba6ab61 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:21:55.249921 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 721/1441] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:21:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:21:57.548726 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2874717-9fdd-4a07-8195-a9f449d6c6bf None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:57.549339 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2874717-9fdd-4a07-8195-a9f449d6c6bf None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:57.549339 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2874717-9fdd-4a07-8195-a9f449d6c6bf None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:57.550623 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 721/1442] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 20:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:21:57.556256 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98c70a-d5de-47e6-a47c-d27a0e5762d3 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:57.557505 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98c70a-d5de-47e6-a47c-d27a0e5762d3 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:57.557992 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98c70a-d5de-47e6-a47c-d27a0e5762d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:57.626849 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-7a98c70a-d5de-47e6-a47c-d27a0e5762d3 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:21:57.628278 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7a98c70a-d5de-47e6-a47c-d27a0e5762d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:57.641933 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:57.731603 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 722/1443] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 20:21:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:21:58.029026 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2874717-9fdd-4a07-8195-a9f449d6c6bf None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:21:58.030629 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d2874717-9fdd-4a07-8195-a9f449d6c6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:58.135714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2874717-9fdd-4a07-8195-a9f449d6c6bf None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:21:58.147879 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:21:58.211642 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd3b9f-fdf1-437c-9b08-813ba6b4e4c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:21:58.212246 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd3b9f-fdf1-437c-9b08-813ba6b4e4c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:21:58.212525 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd3b9f-fdf1-437c-9b08-813ba6b4e4c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:21:58.212850 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd3b9f-fdf1-437c-9b08-813ba6b4e4c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:21:58.214959 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-52dd3b9f-fdf1-437c-9b08-813ba6b4e4c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:21:58.319876 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd3b9f-fdf1-437c-9b08-813ba6b4e4c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:21:58.321329 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd3b9f-fdf1-437c-9b08-813ba6b4e4c2 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:21:58.339775 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 722/1444] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:21:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:22:00.800419 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fbaac-f613-4fec-aac9-f0a2b6983991 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:00.800967 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fbaac-f613-4fec-aac9-f0a2b6983991 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:00.800967 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fbaac-f613-4fec-aac9-f0a2b6983991 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:00.802359 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 723/1445] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 20:22:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:22:00.808106 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ce6d1-fa30-4c03-8f92-cafd00c627a6 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:00.808540 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ce6d1-fa30-4c03-8f92-cafd00c627a6 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:00.808650 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ce6d1-fa30-4c03-8f92-cafd00c627a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:00.855445 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:22:00.874339 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ec4ce6d1-fa30-4c03-8f92-cafd00c627a6 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:22:00.875573 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ec4ce6d1-fa30-4c03-8f92-cafd00c627a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:00.970070 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 723/1446] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 20:22:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:22:01.262567 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248fbaac-f613-4fec-aac9-f0a2b6983991 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:22:01.264123 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-248fbaac-f613-4fec-aac9-f0a2b6983991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:01.365867 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248fbaac-f613-4fec-aac9-f0a2b6983991 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:22:01.405618 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:22:01.438956 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be452762-6239-4ab9-9ea9-89b3e8836490 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:22:01.439557 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be452762-6239-4ab9-9ea9-89b3e8836490 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:01.439836 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be452762-6239-4ab9-9ea9-89b3e8836490 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:01.440121 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be452762-6239-4ab9-9ea9-89b3e8836490 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:01.441710 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-be452762-6239-4ab9-9ea9-89b3e8836490 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:01.548505 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be452762-6239-4ab9-9ea9-89b3e8836490 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:22:01.549907 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be452762-6239-4ab9-9ea9-89b3e8836490 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:22:01.567749 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 724/1447] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:22:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:22:03.817807 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36acd6f-3daf-4327-a8a3-e12e0f3db77a None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:03.818380 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36acd6f-3daf-4327-a8a3-e12e0f3db77a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:03.818380 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36acd6f-3daf-4327-a8a3-e12e0f3db77a None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:03.819682 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 724/1448] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 20:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:22:03.825671 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b9979-20f8-4c66-ab8b-2a08fb770da0 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:03.826055 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b9979-20f8-4c66-ab8b-2a08fb770da0 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:03.826218 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b9979-20f8-4c66-ab8b-2a08fb770da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:03.891119 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-5f3b9979-20f8-4c66-ab8b-2a08fb770da0 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:22:03.892309 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-5f3b9979-20f8-4c66-ab8b-2a08fb770da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:03.912191 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:22:03.987358 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 725/1449] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 20:22:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:22:04.272525 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36acd6f-3daf-4327-a8a3-e12e0f3db77a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:22:04.274113 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b36acd6f-3daf-4327-a8a3-e12e0f3db77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:04.383179 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36acd6f-3daf-4327-a8a3-e12e0f3db77a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:22:04.404047 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:22:04.457626 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9e53c-d822-4e25-9df2-2d320efb8532 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:22:04.458232 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9e53c-d822-4e25-9df2-2d320efb8532 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:04.458510 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9e53c-d822-4e25-9df2-2d320efb8532 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:04.458781 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9e53c-d822-4e25-9df2-2d320efb8532 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:04.460386 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-07a9e53c-d822-4e25-9df2-2d320efb8532 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:04.567204 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a9e53c-d822-4e25-9df2-2d320efb8532 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:22:04.568556 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a9e53c-d822-4e25-9df2-2d320efb8532 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:22:04.586011 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 725/1450] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:22:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 20:22:06.877440 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72cec5-10ff-4d41-b1f1-6cdcea0ae626 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:06.878007 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72cec5-10ff-4d41-b1f1-6cdcea0ae626 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:06.878007 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72cec5-10ff-4d41-b1f1-6cdcea0ae626 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:06.879347 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 726/1451] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 20:22:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:22:06.884581 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcffe8a-ab1a-4b8c-9df7-b9623d4c8f1d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:06.885026 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcffe8a-ab1a-4b8c-9df7-b9623d4c8f1d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:06.885161 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcffe8a-ab1a-4b8c-9df7-b9623d4c8f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:06.948262 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:22:06.951157 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ffcffe8a-ab1a-4b8c-9df7-b9623d4c8f1d None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:22:06.952912 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ffcffe8a-ab1a-4b8c-9df7-b9623d4c8f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:07.047958 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 726/1452] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 20:22:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:22:07.337008 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b72cec5-10ff-4d41-b1f1-6cdcea0ae626 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:22:07.339122 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-3b72cec5-10ff-4d41-b1f1-6cdcea0ae626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:07.443060 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b72cec5-10ff-4d41-b1f1-6cdcea0ae626 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:22:07.491729 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:22:07.516738 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b52379-f4d5-497d-b269-ad1ecbd4295e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:22:07.517371 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b52379-f4d5-497d-b269-ad1ecbd4295e service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:07.517650 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b52379-f4d5-497d-b269-ad1ecbd4295e service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:07.517916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b52379-f4d5-497d-b269-ad1ecbd4295e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:07.519496 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a9b52379-f4d5-497d-b269-ad1ecbd4295e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:07.625722 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b52379-f4d5-497d-b269-ad1ecbd4295e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:22:07.627095 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b52379-f4d5-497d-b269-ad1ecbd4295e service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:22:07.645160 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 727/1453] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:22:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:22:09.855222 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf63a1f-670f-46b9-b174-07b1bdf80f8c None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:09.855736 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf63a1f-670f-46b9-b174-07b1bdf80f8c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:09.855736 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf63a1f-670f-46b9-b174-07b1bdf80f8c None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:09.857011 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 727/1454] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 20:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:22:09.862193 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c254b05-310f-4d67-bba3-bdca64ae8420 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:09.862629 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c254b05-310f-4d67-bba3-bdca64ae8420 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:09.862774 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c254b05-310f-4d67-bba3-bdca64ae8420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:09.928001 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-4c254b05-310f-4d67-bba3-bdca64ae8420 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:22:09.929202 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-4c254b05-310f-4d67-bba3-bdca64ae8420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:09.958877 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:22:10.026677 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 728/1455] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 20:22:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:22:10.305811 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf63a1f-670f-46b9-b174-07b1bdf80f8c None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:22:10.307901 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-dbf63a1f-670f-46b9-b174-07b1bdf80f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:10.411323 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf63a1f-670f-46b9-b174-07b1bdf80f8c None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:22:10.440346 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:22:10.485344 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96673856-9052-4ee0-b877-9110357a92c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:22:10.485945 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96673856-9052-4ee0-b877-9110357a92c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:10.486220 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96673856-9052-4ee0-b877-9110357a92c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:10.486487 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96673856-9052-4ee0-b877-9110357a92c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:10.488063 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-96673856-9052-4ee0-b877-9110357a92c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:10.595133 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96673856-9052-4ee0-b877-9110357a92c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:22:10.596484 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96673856-9052-4ee0-b877-9110357a92c5 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:22:10.614226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 728/1456] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:22:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:22:12.833725 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b618015-74d8-4444-bf6e-a02fa5cb9fc4 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:12.834294 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b618015-74d8-4444-bf6e-a02fa5cb9fc4 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:12.834294 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b618015-74d8-4444-bf6e-a02fa5cb9fc4 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:12.835677 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 729/1457] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 20:22:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:22:12.841592 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0ba49-04a3-49e3-b00c-9489fb75c2b9 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:12.842028 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0ba49-04a3-49e3-b00c-9489fb75c2b9 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:12.842165 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0ba49-04a3-49e3-b00c-9489fb75c2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:12.907239 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a4e0ba49-04a3-49e3-b00c-9489fb75c2b9 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:22:12.908436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a4e0ba49-04a3-49e3-b00c-9489fb75c2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:12.916572 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:22:13.004270 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 729/1458] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 20:22:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:22:13.286556 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b618015-74d8-4444-bf6e-a02fa5cb9fc4 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:22:13.288092 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-3b618015-74d8-4444-bf6e-a02fa5cb9fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:13.390631 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b618015-74d8-4444-bf6e-a02fa5cb9fc4 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:22:13.446423 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:22:13.464489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ee0d1-b9e8-4d9c-b626-e5d971d531cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:22:13.465091 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ee0d1-b9e8-4d9c-b626-e5d971d531cb service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:13.465403 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ee0d1-b9e8-4d9c-b626-e5d971d531cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:13.465674 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ee0d1-b9e8-4d9c-b626-e5d971d531cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:13.467259 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6b4ee0d1-b9e8-4d9c-b626-e5d971d531cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:13.573015 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ee0d1-b9e8-4d9c-b626-e5d971d531cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:22:13.574404 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ee0d1-b9e8-4d9c-b626-e5d971d531cb service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:22:13.592462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 730/1459] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:22:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:22:15.785497 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46faf0-c8eb-4e9a-93b3-bab5eea9a3ab None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:15.786058 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46faf0-c8eb-4e9a-93b3-bab5eea9a3ab None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:15.786058 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46faf0-c8eb-4e9a-93b3-bab5eea9a3ab None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:15.787363 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 730/1460] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 20:22:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:22:15.793086 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb59458-163d-4011-948d-8b120e7b8575 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:15.793520 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb59458-163d-4011-948d-8b120e7b8575 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:15.793679 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb59458-163d-4011-948d-8b120e7b8575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:15.859920 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-0fb59458-163d-4011-948d-8b120e7b8575 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:22:15.861311 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-0fb59458-163d-4011-948d-8b120e7b8575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:15.899138 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:22:15.956616 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 731/1461] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 20:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:22:16.251607 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2771c-cfbc-438c-91b9-7df1613dc62b None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:16.251923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2771c-cfbc-438c-91b9-7df1613dc62b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:16.252195 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2771c-cfbc-438c-91b9-7df1613dc62b None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:16.253554 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 731/1462] 10.222.0.26 () {54 vars in 965 bytes} [Fri Sep 9 20:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:22:16.259295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c296468-796f-41bc-a273-8a33adf4b1a6 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:16.259651 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c296468-796f-41bc-a273-8a33adf4b1a6 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:16.259936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c296468-796f-41bc-a273-8a33adf4b1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:16.334614 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-4c296468-796f-41bc-a273-8a33adf4b1a6 None None] MFA Rules not processed for user `ee5c6e51f2694bf5ae7ea097cf33fb3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:22:16.335806 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-4c296468-796f-41bc-a273-8a33adf4b1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:16.410214 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 732/1463] 10.222.0.26 () {58 vars in 1062 bytes} [Fri Sep 9 20:22:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:22:16.419607 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa2771c-cfbc-438c-91b9-7df1613dc62b None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:22:16.421115 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-faa2771c-cfbc-438c-91b9-7df1613dc62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:16.522712 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa2771c-cfbc-438c-91b9-7df1613dc62b None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:22:16.560715 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:22:16.596343 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe384df-aa07-4e84-905d-1a2fc2691db3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:22:16.596946 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe384df-aa07-4e84-905d-1a2fc2691db3 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:16.597221 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe384df-aa07-4e84-905d-1a2fc2691db3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:16.597521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe384df-aa07-4e84-905d-1a2fc2691db3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:16.599095 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ffe384df-aa07-4e84-905d-1a2fc2691db3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:16.705477 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe384df-aa07-4e84-905d-1a2fc2691db3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:22:16.706892 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe384df-aa07-4e84-905d-1a2fc2691db3 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:22:16.725485 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 732/1464] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:22:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:22:19.284782 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c6a75-15f8-486d-8121-781b598cf974 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:19.285383 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c6a75-15f8-486d-8121-781b598cf974 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:19.285383 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c6a75-15f8-486d-8121-781b598cf974 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:19.286716 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 20:22:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:22:19.292669 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b4548-f049-44bc-8aab-2d7b0fd0e5b9 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:19.293167 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b4548-f049-44bc-8aab-2d7b0fd0e5b9 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:19.293295 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b4548-f049-44bc-8aab-2d7b0fd0e5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:19.358091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-243b4548-f049-44bc-8aab-2d7b0fd0e5b9 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:22:19.359296 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-243b4548-f049-44bc-8aab-2d7b0fd0e5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:19.378212 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:22:19.463800 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 20:22:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:22:19.756729 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c6a75-15f8-486d-8121-781b598cf974 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:22:19.758275 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-126c6a75-15f8-486d-8121-781b598cf974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:19.796436 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:22:19.861140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c6a75-15f8-486d-8121-781b598cf974 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:22:19.934016 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d67391b-2add-4b51-a25f-5a40a2f4163e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:22:19.934577 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67391b-2add-4b51-a25f-5a40a2f4163e service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:19.934855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67391b-2add-4b51-a25f-5a40a2f4163e service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:19.935127 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67391b-2add-4b51-a25f-5a40a2f4163e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:19.936756 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-5d67391b-2add-4b51-a25f-5a40a2f4163e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:20.020547 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:22:20.046989 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d67391b-2add-4b51-a25f-5a40a2f4163e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:22:20.048414 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d67391b-2add-4b51-a25f-5a40a2f4163e service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:22:20.066190 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 734/1467] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:22:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:22:22.299963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd01b4-b921-4567-a0c8-a068295adc35 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:22.300496 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd01b4-b921-4567-a0c8-a068295adc35 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:22.300496 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd01b4-b921-4567-a0c8-a068295adc35 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:22.301770 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 20:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:22:22.307188 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84cce92-cae9-4802-80cf-00e5e02f645b None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:22.307576 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84cce92-cae9-4802-80cf-00e5e02f645b None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:22.307740 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84cce92-cae9-4802-80cf-00e5e02f645b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:22.372778 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-d84cce92-cae9-4802-80cf-00e5e02f645b None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:22:22.374295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-d84cce92-cae9-4802-80cf-00e5e02f645b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:22.474481 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 20:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:22:22.757985 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd01b4-b921-4567-a0c8-a068295adc35 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:22:22.759510 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-bdfd01b4-b921-4567-a0c8-a068295adc35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:22.861259 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd01b4-b921-4567-a0c8-a068295adc35 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:22:22.910292 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:22:22.935949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af438f-dbac-4877-851f-553c92c408ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:22:22.936501 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af438f-dbac-4877-851f-553c92c408ca service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:22.936781 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af438f-dbac-4877-851f-553c92c408ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:22.937051 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af438f-dbac-4877-851f-553c92c408ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:22.938674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b9af438f-dbac-4877-851f-553c92c408ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:23.045630 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af438f-dbac-4877-851f-553c92c408ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:22:23.047055 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af438f-dbac-4877-851f-553c92c408ca service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:22:23.066622 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 735/1470] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:22:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:22:32.472955 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=81 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:22:33.064218 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=66 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:22:42.923780 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972395cb-c8f1-4a9c-9e69-de5b6e2647f3 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:42.924347 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972395cb-c8f1-4a9c-9e69-de5b6e2647f3 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:42.924347 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972395cb-c8f1-4a9c-9e69-de5b6e2647f3 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:42.925627 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 736/1471] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:22:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:22:42.931434 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7e5c8-8020-4489-bba8-5ce5165550ed None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:42.931859 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7e5c8-8020-4489-bba8-5ce5165550ed None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:42.931985 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7e5c8-8020-4489-bba8-5ce5165550ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:42.996709 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-c2b7e5c8-8020-4489-bba8-5ce5165550ed None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:22:42.998188 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c2b7e5c8-8020-4489-bba8-5ce5165550ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:43.093727 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 736/1472] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:22:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:22:43.393445 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750eecb-4863-4d7b-b9ea-01527f86a88c None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:43.393730 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750eecb-4863-4d7b-b9ea-01527f86a88c None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:43.394003 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750eecb-4863-4d7b-b9ea-01527f86a88c None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:43.395345 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 737/1473] 10.222.0.26 () {54 vars in 966 bytes} [Fri Sep 9 20:22:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:22:43.400888 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141b6bd-e533-436d-ad68-ccb4f7642765 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:43.401171 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141b6bd-e533-436d-ad68-ccb4f7642765 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:43.401472 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141b6bd-e533-436d-ad68-ccb4f7642765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:43.475177 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-d141b6bd-e533-436d-ad68-ccb4f7642765 None None] MFA Rules not processed for user `5298486310004b63b51116dbc747e31d`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:22:43.476710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d141b6bd-e533-436d-ad68-ccb4f7642765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:43.504341 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:22:43.559945 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 737/1474] 10.222.0.26 () {58 vars in 1063 bytes} [Fri Sep 9 20:22:43 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:22:43.567643 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a750eecb-4863-4d7b-b9ea-01527f86a88c None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:22:43.569170 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a750eecb-4863-4d7b-b9ea-01527f86a88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:43.674809 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a750eecb-4863-4d7b-b9ea-01527f86a88c None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:22:43.746636 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0143974a-4063-4021-b73d-fb50169bde8d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5298486310004b63b51116dbc747e31d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:22:43.747233 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0143974a-4063-4021-b73d-fb50169bde8d service glance] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:43.747764 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0143974a-4063-4021-b73d-fb50169bde8d service glance] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:43.748032 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0143974a-4063-4021-b73d-fb50169bde8d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:43.749640 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-0143974a-4063-4021-b73d-fb50169bde8d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:43.802868 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:22:43.863049 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0143974a-4063-4021-b73d-fb50169bde8d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:22:43.864478 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0143974a-4063-4021-b73d-fb50169bde8d service glance] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:22:43.883401 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 738/1475] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:22:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:22:46.097990 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c12630-3882-471d-8348-a8d216d86d50 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:46.098551 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c12630-3882-471d-8348-a8d216d86d50 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:46.098821 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c12630-3882-471d-8348-a8d216d86d50 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:46.100156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 738/1476] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:22:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:22:46.105944 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335b53a-e302-44cc-92a8-31a22a81469a None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:46.106401 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335b53a-e302-44cc-92a8-31a22a81469a None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:46.106510 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335b53a-e302-44cc-92a8-31a22a81469a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:46.171166 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-9335b53a-e302-44cc-92a8-31a22a81469a None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:22:46.172346 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-9335b53a-e302-44cc-92a8-31a22a81469a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:46.266953 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 739/1477] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:22:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:22:46.541807 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c12630-3882-471d-8348-a8d216d86d50 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:22:46.543264 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a2c12630-3882-471d-8348-a8d216d86d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:46.670121 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c12630-3882-471d-8348-a8d216d86d50 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:22:46.720024 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:22:46.764624 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5735e16-2237-4c34-8d65-386873300edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:22:46.765231 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5735e16-2237-4c34-8d65-386873300edc admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:46.765541 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5735e16-2237-4c34-8d65-386873300edc admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:46.765816 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5735e16-2237-4c34-8d65-386873300edc admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:46.766454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5735e16-2237-4c34-8d65-386873300edc admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:22:46.768159 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: WARNING py.warnings [None req-e5735e16-2237-4c34-8d65-386873300edc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 20:22:46.768159 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: warnings.warn(msg) Sep 09 20:22:46.768159 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: Sep 09 20:22:46.769325 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5735e16-2237-4c34-8d65-386873300edc admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:22:46.787876 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 739/1478] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Sep 9 20:22:46 2022] GET /identity/v3/projects => generated 2553 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:22:48.927048 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656bfe4-eebf-4631-ae09-456c73eb4c0e None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:48.927614 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656bfe4-eebf-4631-ae09-456c73eb4c0e None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:48.927614 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656bfe4-eebf-4631-ae09-456c73eb4c0e None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:48.928930 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 740/1479] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:22:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:22:48.934475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ccbe3c-d6a5-449c-a956-81af64387766 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:48.934911 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ccbe3c-d6a5-449c-a956-81af64387766 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:48.935097 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ccbe3c-d6a5-449c-a956-81af64387766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:49.000750 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-89ccbe3c-d6a5-449c-a956-81af64387766 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:22:49.001994 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-89ccbe3c-d6a5-449c-a956-81af64387766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:49.099647 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 740/1480] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:22:49.398681 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3656bfe4-eebf-4631-ae09-456c73eb4c0e None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:22:49.400279 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-3656bfe4-eebf-4631-ae09-456c73eb4c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:49.410079 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:22:49.544823 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3656bfe4-eebf-4631-ae09-456c73eb4c0e None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:22:49.622539 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:22:49.654916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665b0d9-cc18-4cb2-854e-2d3f52d14234 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:22:49.655849 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665b0d9-cc18-4cb2-854e-2d3f52d14234 service nova] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:49.656128 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665b0d9-cc18-4cb2-854e-2d3f52d14234 service nova] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:49.656398 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665b0d9-cc18-4cb2-854e-2d3f52d14234 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:49.658017 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-8665b0d9-cc18-4cb2-854e-2d3f52d14234 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:49.746200 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8665b0d9-cc18-4cb2-854e-2d3f52d14234 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:22:49.748067 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8665b0d9-cc18-4cb2-854e-2d3f52d14234 service nova] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:22:49.766277 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 741/1481] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:22:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:22:52.084607 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e56d0f7-7502-4c9c-9e1f-b006d9d4c5ef None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:52.085692 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e56d0f7-7502-4c9c-9e1f-b006d9d4c5ef None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:52.085692 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e56d0f7-7502-4c9c-9e1f-b006d9d4c5ef None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:52.087059 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 741/1482] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:22:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:22:52.092429 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70c28c-9940-4881-94f1-9dfe51803d4e None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:52.092853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70c28c-9940-4881-94f1-9dfe51803d4e None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:52.092982 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70c28c-9940-4881-94f1-9dfe51803d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:52.159105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-6d70c28c-9940-4881-94f1-9dfe51803d4e None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:22:52.160288 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6d70c28c-9940-4881-94f1-9dfe51803d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:52.205541 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:22:52.257584 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 742/1483] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:22:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:22:52.554529 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e56d0f7-7502-4c9c-9e1f-b006d9d4c5ef None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:22:52.556075 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8e56d0f7-7502-4c9c-9e1f-b006d9d4c5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:52.580046 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:22:52.661416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e56d0f7-7502-4c9c-9e1f-b006d9d4c5ef None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:22:52.736055 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d29ccb7-9971-4041-9e00-92777fe58b95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:22:52.736670 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29ccb7-9971-4041-9e00-92777fe58b95 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:52.736949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29ccb7-9971-4041-9e00-92777fe58b95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:52.737219 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29ccb7-9971-4041-9e00-92777fe58b95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:52.738842 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2d29ccb7-9971-4041-9e00-92777fe58b95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:52.812916 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:22:52.847763 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d29ccb7-9971-4041-9e00-92777fe58b95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:22:52.849193 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d29ccb7-9971-4041-9e00-92777fe58b95 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:22:52.867263 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 742/1484] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:22:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:22:55.054939 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee0553-344b-439d-ba82-1de759213c89 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:55.055543 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee0553-344b-439d-ba82-1de759213c89 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:55.055543 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee0553-344b-439d-ba82-1de759213c89 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:55.056829 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 743/1485] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:22:55.062565 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024634e9-0346-4db4-a515-62ba4598049e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:55.063087 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024634e9-0346-4db4-a515-62ba4598049e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:55.063193 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024634e9-0346-4db4-a515-62ba4598049e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:55.128726 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-024634e9-0346-4db4-a515-62ba4598049e None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:22:55.129948 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-024634e9-0346-4db4-a515-62ba4598049e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:55.224929 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 743/1486] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:22:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:22:55.523286 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee0553-344b-439d-ba82-1de759213c89 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:22:55.524819 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7eee0553-344b-439d-ba82-1de759213c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:55.625275 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee0553-344b-439d-ba82-1de759213c89 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:22:55.702739 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:22:55.704277 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c611b3-fb48-4f3d-9080-b4c8c733c02b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:22:55.704851 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c611b3-fb48-4f3d-9080-b4c8c733c02b service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:55.705358 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c611b3-fb48-4f3d-9080-b4c8c733c02b service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:55.705793 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c611b3-fb48-4f3d-9080-b4c8c733c02b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:55.707557 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-c7c611b3-fb48-4f3d-9080-b4c8c733c02b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:55.813905 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c611b3-fb48-4f3d-9080-b4c8c733c02b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:22:55.815260 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c611b3-fb48-4f3d-9080-b4c8c733c02b service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:22:55.832750 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 744/1487] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:22:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:22:58.099484 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d7d5d-05c5-4592-aeb3-c820d22693a2 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:58.100063 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d7d5d-05c5-4592-aeb3-c820d22693a2 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:58.100063 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d7d5d-05c5-4592-aeb3-c820d22693a2 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:58.101433 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 744/1488] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:22:58.107237 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac011c4-0dd7-43d4-9895-0290a61df62a None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:58.107676 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac011c4-0dd7-43d4-9895-0290a61df62a None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:58.107802 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac011c4-0dd7-43d4-9895-0290a61df62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:58.175892 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-9ac011c4-0dd7-43d4-9895-0290a61df62a None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:22:58.177079 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-9ac011c4-0dd7-43d4-9895-0290a61df62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:58.228905 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:22:58.273009 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 745/1489] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:22:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:22:58.563180 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d7d5d-05c5-4592-aeb3-c820d22693a2 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:22:58.564729 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-596d7d5d-05c5-4592-aeb3-c820d22693a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:58.601070 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:22:58.669960 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d7d5d-05c5-4592-aeb3-c820d22693a2 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:22:58.744350 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c4393-f410-404c-8798-7b135d2d16bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:22:58.744961 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c4393-f410-404c-8798-7b135d2d16bd service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:22:58.745258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c4393-f410-404c-8798-7b135d2d16bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:22:58.745539 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c4393-f410-404c-8798-7b135d2d16bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:22:58.747386 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5e2c4393-f410-404c-8798-7b135d2d16bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:22:58.825210 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:22:58.855236 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c4393-f410-404c-8798-7b135d2d16bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:22:58.856665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c4393-f410-404c-8798-7b135d2d16bd service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:22:58.874547 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 745/1490] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:22:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:23:01.196047 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aee541-752f-4ef5-9b4a-c3ee94fb64a2 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:23:01.196611 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aee541-752f-4ef5-9b4a-c3ee94fb64a2 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:23:01.196611 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aee541-752f-4ef5-9b4a-c3ee94fb64a2 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:23:01.197963 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 746/1491] 10.222.0.26 () {54 vars in 927 bytes} [Fri Sep 9 20:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:23:01.203450 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee44a76-7c25-4e54-be8d-51393491c891 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:23:01.203864 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee44a76-7c25-4e54-be8d-51393491c891 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:23:01.204015 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee44a76-7c25-4e54-be8d-51393491c891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:23:01.269774 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-6ee44a76-7c25-4e54-be8d-51393491c891 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:23:01.270983 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6ee44a76-7c25-4e54-be8d-51393491c891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:23:01.367572 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 746/1492] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Sep 9 20:23:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:23:01.656794 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aee541-752f-4ef5-9b4a-c3ee94fb64a2 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:23:01.658445 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-b5aee541-752f-4ef5-9b4a-c3ee94fb64a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:23:01.762056 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aee541-752f-4ef5-9b4a-c3ee94fb64a2 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:23:01.835426 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e074cee-db0b-4944-a244-1d84a4ef1e95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:23:01.836028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e074cee-db0b-4944-a244-1d84a4ef1e95 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:23:01.836311 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e074cee-db0b-4944-a244-1d84a4ef1e95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:23:01.836584 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e074cee-db0b-4944-a244-1d84a4ef1e95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:23:01.838201 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6e074cee-db0b-4944-a244-1d84a4ef1e95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:23:01.846771 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:23:01.945789 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e074cee-db0b-4944-a244-1d84a4ef1e95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:23:01.947225 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e074cee-db0b-4944-a244-1d84a4ef1e95 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:23:01.965026 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 747/1493] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:23:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:23:11.365430 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=82 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:23:11.962784 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=50 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:24:29.027330 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04999ed6-e3ae-4038-a182-c1e34377511f None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:24:29.027908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04999ed6-e3ae-4038-a182-c1e34377511f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:24:29.027908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04999ed6-e3ae-4038-a182-c1e34377511f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:24:29.107055 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-04999ed6-e3ae-4038-a182-c1e34377511f None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:24:29.108280 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-04999ed6-e3ae-4038-a182-c1e34377511f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:24:29.208156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 747/1494] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:24:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:24:29.216183 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f30f2-47fa-41f4-8732-616bdcbdb446 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:24:29.216668 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f30f2-47fa-41f4-8732-616bdcbdb446 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:24:29.216768 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f30f2-47fa-41f4-8732-616bdcbdb446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:24:29.291856 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-ec2f30f2-47fa-41f4-8732-616bdcbdb446 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:24:29.292987 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-ec2f30f2-47fa-41f4-8732-616bdcbdb446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:24:29.387835 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 748/1495] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:24:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:24:29.392606 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04999ed6-e3ae-4038-a182-c1e34377511f None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:24:29.393898 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-04999ed6-e3ae-4038-a182-c1e34377511f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:24:29.405164 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04999ed6-e3ae-4038-a182-c1e34377511f None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:24:29.469402 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:24:29.503547 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056d0ae-23fe-43e5-b473-cdf8f4e3a5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:24:29.504151 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056d0ae-23fe-43e5-b473-cdf8f4e3a5be admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:24:29.504429 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056d0ae-23fe-43e5-b473-cdf8f4e3a5be admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:24:29.504703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056d0ae-23fe-43e5-b473-cdf8f4e3a5be admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:24:29.505368 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3056d0ae-23fe-43e5-b473-cdf8f4e3a5be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:24:29.505666 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3056d0ae-23fe-43e5-b473-cdf8f4e3a5be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:24:29.506664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: WARNING py.warnings [None req-3056d0ae-23fe-43e5-b473-cdf8f4e3a5be admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 20:24:29.506664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: warnings.warn(msg) Sep 09 20:24:29.506664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: Sep 09 20:24:29.507542 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3056d0ae-23fe-43e5-b473-cdf8f4e3a5be admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:24:29.518478 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 748/1496] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:24:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:24:29.526702 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f30f2-47fa-41f4-8732-616bdcbdb446 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:24:29.536122 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f30f2-47fa-41f4-8732-616bdcbdb446 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:24:29.596987 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:24:29.632076 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6d091-5a86-449d-a4a3-7fe13645adbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:24:29.632679 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6d091-5a86-449d-a4a3-7fe13645adbf admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:24:29.632960 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6d091-5a86-449d-a4a3-7fe13645adbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:24:29.633231 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6d091-5a86-449d-a4a3-7fe13645adbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:24:29.633896 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d6d091-5a86-449d-a4a3-7fe13645adbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:24:29.635683 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: WARNING py.warnings [None req-34d6d091-5a86-449d-a4a3-7fe13645adbf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 20:24:29.635683 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: warnings.warn(msg) Sep 09 20:24:29.635683 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: Sep 09 20:24:29.636895 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d6d091-5a86-449d-a4a3-7fe13645adbf admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:24:29.808664 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 749/1497] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:24:29 2022] POST /identity/v3/projects => generated 377 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:24:29.812904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056d0ae-23fe-43e5-b473-cdf8f4e3a5be admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:24:29.814526 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3056d0ae-23fe-43e5-b473-cdf8f4e3a5be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:24:29.847455 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056d0ae-23fe-43e5-b473-cdf8f4e3a5be admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:24:29.939169 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb19c7-c5f6-40b0-9d2e-f15d474e423a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:24:29.939768 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb19c7-c5f6-40b0-9d2e-f15d474e423a admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:24:29.940045 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb19c7-c5f6-40b0-9d2e-f15d474e423a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:24:29.940316 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb19c7-c5f6-40b0-9d2e-f15d474e423a admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:24:29.940951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bb19c7-c5f6-40b0-9d2e-f15d474e423a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:24:29.941983 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: WARNING py.warnings [None req-69bb19c7-c5f6-40b0-9d2e-f15d474e423a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 20:24:29.941983 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: warnings.warn(msg) Sep 09 20:24:29.941983 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: Sep 09 20:24:29.943186 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bb19c7-c5f6-40b0-9d2e-f15d474e423a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:24:29.984600 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 749/1498] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:24:29 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:24:29.989328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6d091-5a86-449d-a4a3-7fe13645adbf admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:24:29.990831 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-34d6d091-5a86-449d-a4a3-7fe13645adbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:24:30.052552 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6d091-5a86-449d-a4a3-7fe13645adbf admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:24:30.134135 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:24:30.145790 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea85d4-70c7-41a3-936c-137dd8d9412b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:24:30.146398 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea85d4-70c7-41a3-936c-137dd8d9412b admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:24:30.146679 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea85d4-70c7-41a3-936c-137dd8d9412b admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:24:30.146953 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea85d4-70c7-41a3-936c-137dd8d9412b admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:24:30.147600 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ea85d4-70c7-41a3-936c-137dd8d9412b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:24:30.154083 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: WARNING py.warnings [None req-b5ea85d4-70c7-41a3-936c-137dd8d9412b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 20:24:30.154083 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: warnings.warn(msg) Sep 09 20:24:30.154083 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: Sep 09 20:24:30.156031 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ea85d4-70c7-41a3-936c-137dd8d9412b admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:24:30.169014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 750/1499] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:24:29 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:24:30.173086 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb19c7-c5f6-40b0-9d2e-f15d474e423a admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:24:30.174933 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-69bb19c7-c5f6-40b0-9d2e-f15d474e423a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:24:30.186049 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb19c7-c5f6-40b0-9d2e-f15d474e423a admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:24:30.210759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:24:30.280179 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dec663f-0011-468b-b9df-21bc0fdb39c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:24:30.280771 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec663f-0011-468b-b9df-21bc0fdb39c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:24:30.281047 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec663f-0011-468b-b9df-21bc0fdb39c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:24:30.281343 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec663f-0011-468b-b9df-21bc0fdb39c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:24:30.281968 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dec663f-0011-468b-b9df-21bc0fdb39c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:24:30.283480 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: WARNING py.warnings [None req-5dec663f-0011-468b-b9df-21bc0fdb39c5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 20:24:30.283480 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: warnings.warn(msg) Sep 09 20:24:30.283480 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: Sep 09 20:24:30.284351 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dec663f-0011-468b-b9df-21bc0fdb39c5 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:24:30.296346 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 750/1500] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:24:30.300582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea85d4-70c7-41a3-936c-137dd8d9412b admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:24:30.302140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-b5ea85d4-70c7-41a3-936c-137dd8d9412b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:24:30.335211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea85d4-70c7-41a3-936c-137dd8d9412b admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:24:30.427200 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab312dcb-0c18-4869-b23f-1edd5cfd6e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:24:30.427779 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab312dcb-0c18-4869-b23f-1edd5cfd6e83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:24:30.428057 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab312dcb-0c18-4869-b23f-1edd5cfd6e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:24:30.428328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab312dcb-0c18-4869-b23f-1edd5cfd6e83 admin admin] PATH_INFO: `/v3/projects/20368ebedcf645788364065a70951a5b/users/f87e879e64ef405095e5e5a2ba1f18a3/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:24:30.456323 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab312dcb-0c18-4869-b23f-1edd5cfd6e83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20368ebedcf645788364065a70951a5b, user_id=f87e879e64ef405095e5e5a2ba1f18a3, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:24:30.457376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: WARNING py.warnings [None req-ab312dcb-0c18-4869-b23f-1edd5cfd6e83 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 20:24:30.457376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: warnings.warn(msg) Sep 09 20:24:30.457376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: Sep 09 20:24:30.459302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab312dcb-0c18-4869-b23f-1edd5cfd6e83 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:24:30.483550 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 751/1501] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:24:30 2022] PUT /identity/v3/projects/20368ebedcf645788364065a70951a5b/users/f87e879e64ef405095e5e5a2ba1f18a3/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:24:30.496060 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1e488-f5a1-42f4-8de4-cbfbff5ec9af None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:24:30.496349 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1e488-f5a1-42f4-8de4-cbfbff5ec9af None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:24:30.496625 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1e488-f5a1-42f4-8de4-cbfbff5ec9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:24:30.557161 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-13e1e488-f5a1-42f4-8de4-cbfbff5ec9af None None] MFA Rules not processed for user `f87e879e64ef405095e5e5a2ba1f18a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:24:30.558409 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-13e1e488-f5a1-42f4-8de4-cbfbff5ec9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:24:30.634725 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 751/1502] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:24:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:24:30.648730 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338cd114-8085-4ace-ba8a-0ed6e68fe6b5 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:24:30.649037 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338cd114-8085-4ace-ba8a-0ed6e68fe6b5 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:24:30.649329 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338cd114-8085-4ace-ba8a-0ed6e68fe6b5 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:24:30.650646 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 752/1503] 10.222.0.26 () {54 vars in 964 bytes} [Fri Sep 9 20:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:24:30.655427 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272c373-2acb-4fa9-9c99-b79f1a3cc253 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:24:30.655717 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272c373-2acb-4fa9-9c99-b79f1a3cc253 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:24:30.655997 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272c373-2acb-4fa9-9c99-b79f1a3cc253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:24:30.687445 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:24:30.731926 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-6272c373-2acb-4fa9-9c99-b79f1a3cc253 None None] MFA Rules not processed for user `11d86d2df1cb485592e8a5d7387131ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:24:30.733130 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6272c373-2acb-4fa9-9c99-b79f1a3cc253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:24:30.813222 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 752/1504] 10.222.0.26 () {58 vars in 1061 bytes} [Fri Sep 9 20:24:30 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:24:30.821013 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338cd114-8085-4ace-ba8a-0ed6e68fe6b5 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:24:30.822584 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-338cd114-8085-4ace-ba8a-0ed6e68fe6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:24:30.872278 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:24:30.926589 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338cd114-8085-4ace-ba8a-0ed6e68fe6b5 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:24:30.997988 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6820d6b1-b156-4af4-b741-6334faeb791a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:24:30.998585 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6820d6b1-b156-4af4-b741-6334faeb791a service cinder] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:24:30.998860 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6820d6b1-b156-4af4-b741-6334faeb791a service cinder] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:24:30.999127 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6820d6b1-b156-4af4-b741-6334faeb791a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:24:31.000703 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6820d6b1-b156-4af4-b741-6334faeb791a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:24:31.096807 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6820d6b1-b156-4af4-b741-6334faeb791a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:24:31.098264 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6820d6b1-b156-4af4-b741-6334faeb791a service cinder] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:24:31.101673 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:24:31.116215 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 753/1505] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:24:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:24:31.161741 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ba1fb-c0ce-43f4-81ff-247b88bd2e61 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:24:31.162195 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ba1fb-c0ce-43f4-81ff-247b88bd2e61 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:24:31.162602 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ba1fb-c0ce-43f4-81ff-247b88bd2e61 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:24:31.163849 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 753/1506] 10.222.0.26 () {52 vars in 830 bytes} [Fri Sep 9 20:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:24:31.167724 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6820d6b1-b156-4af4-b741-6334faeb791a service cinder] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:24:31.169189 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6820d6b1-b156-4af4-b741-6334faeb791a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:24:31.267329 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6820d6b1-b156-4af4-b741-6334faeb791a service cinder] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:24:31.357102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1abb7d-bfaf-4b40-8b22-aab0661d343c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:24:31.357816 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1abb7d-bfaf-4b40-8b22-aab0661d343c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:24:31.358012 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1abb7d-bfaf-4b40-8b22-aab0661d343c admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:24:31.358279 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1abb7d-bfaf-4b40-8b22-aab0661d343c admin admin] PATH_INFO: `/v3/users/f87e879e64ef405095e5e5a2ba1f18a3` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:24:31.371525 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1abb7d-bfaf-4b40-8b22-aab0661d343c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f87e879e64ef405095e5e5a2ba1f18a3)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:24:31.372533 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: WARNING py.warnings [None req-8e1abb7d-bfaf-4b40-8b22-aab0661d343c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 20:24:31.372533 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: warnings.warn(msg) Sep 09 20:24:31.372533 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: Sep 09 20:24:31.373754 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1abb7d-bfaf-4b40-8b22-aab0661d343c admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:24:31.383231 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:24:31.439772 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-8e1abb7d-bfaf-4b40-8b22-aab0661d343c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f87e879e64ef405095e5e5a2ba1f18a3'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:24:31.462041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-8e1abb7d-bfaf-4b40-8b22-aab0661d343c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f87e879e64ef405095e5e5a2ba1f18a3'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:24:31.476046 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-8e1abb7d-bfaf-4b40-8b22-aab0661d343c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f87e879e64ef405095e5e5a2ba1f18a3'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:24:31.485000 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-8e1abb7d-bfaf-4b40-8b22-aab0661d343c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f87e879e64ef405095e5e5a2ba1f18a3'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:24:31.485496 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-8e1abb7d-bfaf-4b40-8b22-aab0661d343c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f87e879e64ef405095e5e5a2ba1f18a3'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:24:31.491960 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 754/1507] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:24:31 2022] DELETE /identity/v3/users/f87e879e64ef405095e5e5a2ba1f18a3 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:24:31.496995 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fb5d1-e15e-41af-af20-ae035ecf18cc None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:24:31.497519 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fb5d1-e15e-41af-af20-ae035ecf18cc None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:24:31.497617 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fb5d1-e15e-41af-af20-ae035ecf18cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:24:31.554454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-139fb5d1-e15e-41af-af20-ae035ecf18cc None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:24:31.555575 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-139fb5d1-e15e-41af-af20-ae035ecf18cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:24:31.629064 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:24:31.642597 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 754/1508] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:24:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:24:31.652075 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1abb7d-bfaf-4b40-8b22-aab0661d343c admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:24:31.653361 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-8e1abb7d-bfaf-4b40-8b22-aab0661d343c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:24:31.753134 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1abb7d-bfaf-4b40-8b22-aab0661d343c admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:24:31.807473 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:24:31.826739 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd2b988-d6c7-44e0-aa8e-a76536e93cc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:24:31.827353 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2b988-d6c7-44e0-aa8e-a76536e93cc0 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:24:31.827628 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2b988-d6c7-44e0-aa8e-a76536e93cc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:24:31.827899 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2b988-d6c7-44e0-aa8e-a76536e93cc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:24:31.830053 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-5bd2b988-d6c7-44e0-aa8e-a76536e93cc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:24:31.936978 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd2b988-d6c7-44e0-aa8e-a76536e93cc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:24:31.938444 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd2b988-d6c7-44e0-aa8e-a76536e93cc0 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:24:31.959099 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 755/1509] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:24:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:24:32.310078 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139fb5d1-e15e-41af-af20-ae035ecf18cc None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:24:32.311383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-139fb5d1-e15e-41af-af20-ae035ecf18cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:24:32.323425 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139fb5d1-e15e-41af-af20-ae035ecf18cc None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:24:32.423765 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a447c39-5fbf-4277-9138-afe16740dfde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:24:32.424404 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a447c39-5fbf-4277-9138-afe16740dfde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:24:32.424682 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a447c39-5fbf-4277-9138-afe16740dfde admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:24:32.424955 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a447c39-5fbf-4277-9138-afe16740dfde admin admin] PATH_INFO: `/v3/projects/20368ebedcf645788364065a70951a5b` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:24:32.435484 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a447c39-5fbf-4277-9138-afe16740dfde admin admin] RBAC: Authorizing `identity:delete_project(project_id=20368ebedcf645788364065a70951a5b)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:24:32.436493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: WARNING py.warnings [None req-6a447c39-5fbf-4277-9138-afe16740dfde admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 20:24:32.436493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: warnings.warn(msg) Sep 09 20:24:32.436493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: Sep 09 20:24:32.437725 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a447c39-5fbf-4277-9138-afe16740dfde admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:24:32.494297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6a447c39-5fbf-4277-9138-afe16740dfde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20368ebedcf645788364065a70951a5b', 'request_id': 'req-6a447c39-5fbf-4277-9138-afe16740dfde'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:24:32.500655 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6a447c39-5fbf-4277-9138-afe16740dfde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20368ebedcf645788364065a70951a5b', 'request_id': 'req-6a447c39-5fbf-4277-9138-afe16740dfde'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:24:32.513887 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6a447c39-5fbf-4277-9138-afe16740dfde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20368ebedcf645788364065a70951a5b', 'request_id': 'req-6a447c39-5fbf-4277-9138-afe16740dfde'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:24:32.541865 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6a447c39-5fbf-4277-9138-afe16740dfde admin admin] The token cache is being invalidate because project 20368ebedcf645788364065a70951a5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:24:32.542156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6a447c39-5fbf-4277-9138-afe16740dfde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:24:32.542583 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6a447c39-5fbf-4277-9138-afe16740dfde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:24:32.543983 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 755/1510] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:24:32 2022] DELETE /identity/v3/projects/20368ebedcf645788364065a70951a5b => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:24:41.956858 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=62 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:24:42.539797 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:26:01.560406 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a7db7d-1ae5-4082-81f3-22d8184af414 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:26:01.561023 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a7db7d-1ae5-4082-81f3-22d8184af414 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:26:01.561023 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a7db7d-1ae5-4082-81f3-22d8184af414 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:26:01.562325 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 20:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:26:01.567210 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c736e42c-1f22-4e6a-ae11-b0d910eb2724 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:26:01.567670 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c736e42c-1f22-4e6a-ae11-b0d910eb2724 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:26:01.567789 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c736e42c-1f22-4e6a-ae11-b0d910eb2724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:26:01.647579 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-c736e42c-1f22-4e6a-ae11-b0d910eb2724 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:26:01.648778 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c736e42c-1f22-4e6a-ae11-b0d910eb2724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:26:01.752309 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 20:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:26:01.912898 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a7db7d-1ae5-4082-81f3-22d8184af414 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:26:01.915043 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-c2a7db7d-1ae5-4082-81f3-22d8184af414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:26:02.032620 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a7db7d-1ae5-4082-81f3-22d8184af414 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:26:02.038248 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e0ef10-e973-4369-b35f-f1a30d792c64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:26:02.038860 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0ef10-e973-4369-b35f-f1a30d792c64 service nova] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:26:02.039145 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0ef10-e973-4369-b35f-f1a30d792c64 service nova] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:26:02.039463 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0ef10-e973-4369-b35f-f1a30d792c64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:26:02.047767 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e0ef10-e973-4369-b35f-f1a30d792c64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:26:02.049378 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e0ef10-e973-4369-b35f-f1a30d792c64 service nova] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:26:02.059265 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 757/1513] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:26:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:26:03.142358 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0e497-9449-415f-8aee-e25c1052a4e6 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:26:03.142903 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0e497-9449-415f-8aee-e25c1052a4e6 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:26:03.143020 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0e497-9449-415f-8aee-e25c1052a4e6 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:26:03.144254 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 757/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 20:26:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:26:03.151092 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde4c3d-8361-4dfc-ba7b-dd64a25f61b1 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:26:03.151534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde4c3d-8361-4dfc-ba7b-dd64a25f61b1 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:26:03.151643 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde4c3d-8361-4dfc-ba7b-dd64a25f61b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:26:03.171206 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-1fde4c3d-8361-4dfc-ba7b-dd64a25f61b1 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:26:03.172387 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-1fde4c3d-8361-4dfc-ba7b-dd64a25f61b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:26:03.180020 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 758/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 20:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:26:03.332412 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd0e497-9449-415f-8aee-e25c1052a4e6 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:26:03.333372 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2e6cd-6861-41f0-8eba-dc05c29c3303 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:26:03.333687 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2e6cd-6861-41f0-8eba-dc05c29c3303 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:26:03.333966 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2e6cd-6861-41f0-8eba-dc05c29c3303 None None] PATH_INFO: `/v3` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:26:03.335226 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 759/1516] 10.222.0.25 () {54 vars in 925 bytes} [Fri Sep 9 20:26:03 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:26:03.341589 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ffe33-e430-4ed2-b424-ab528a069738 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:26:03.341868 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ffe33-e430-4ed2-b424-ab528a069738 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:26:03.342143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ffe33-e430-4ed2-b424-ab528a069738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:26:03.347267 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd0e497-9449-415f-8aee-e25c1052a4e6 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:26:03.352834 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564be98-d1ea-4ff2-a332-8555e42fc329 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:26:03.353457 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564be98-d1ea-4ff2-a332-8555e42fc329 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:26:03.353733 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564be98-d1ea-4ff2-a332-8555e42fc329 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:26:03.354001 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564be98-d1ea-4ff2-a332-8555e42fc329 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:26:03.362203 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564be98-d1ea-4ff2-a332-8555e42fc329 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:26:03.363765 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564be98-d1ea-4ff2-a332-8555e42fc329 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:26:03.373828 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 758/1517] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:26:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:26:03.404191 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-f91ffe33-e430-4ed2-b424-ab528a069738 None None] MFA Rules not processed for user `26d24cf0a21d491082b0c2f11e749446`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:26:03.405361 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-f91ffe33-e430-4ed2-b424-ab528a069738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:26:03.463273 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:26:03.483194 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 760/1518] 10.222.0.25 () {58 vars in 1015 bytes} [Fri Sep 9 20:26:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:26:07.110407 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b51fea-adb6-43db-8adc-3a4556a4fd9f None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:26:07.110852 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b51fea-adb6-43db-8adc-3a4556a4fd9f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:26:07.110962 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b51fea-adb6-43db-8adc-3a4556a4fd9f None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:26:07.112281 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 759/1519] 10.222.0.26 () {54 vars in 961 bytes} [Fri Sep 9 20:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:26:07.117265 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e439738-0991-4e40-93fb-d507922ce491 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:26:07.117696 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e439738-0991-4e40-93fb-d507922ce491 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:26:07.117827 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e439738-0991-4e40-93fb-d507922ce491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:26:07.137206 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-9e439738-0991-4e40-93fb-d507922ce491 None None] MFA Rules not processed for user `26d24cf0a21d491082b0c2f11e749446`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:26:07.138416 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-9e439738-0991-4e40-93fb-d507922ce491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:26:07.147182 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 761/1520] 10.222.0.26 () {58 vars in 1058 bytes} [Fri Sep 9 20:26:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:26:07.154398 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b51fea-adb6-43db-8adc-3a4556a4fd9f None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:26:07.167796 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b51fea-adb6-43db-8adc-3a4556a4fd9f None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:26:07.172656 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050fb1b1-1e97-498c-90e2-40423f11425b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26d24cf0a21d491082b0c2f11e749446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:26:07.173234 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fb1b1-1e97-498c-90e2-40423f11425b service placement] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:26:07.173541 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fb1b1-1e97-498c-90e2-40423f11425b service placement] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:26:07.173812 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fb1b1-1e97-498c-90e2-40423f11425b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:26:07.181835 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050fb1b1-1e97-498c-90e2-40423f11425b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:26:07.183421 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050fb1b1-1e97-498c-90e2-40423f11425b service placement] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:26:07.192973 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 760/1521] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:26:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:26:14.857593 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a69db-1592-4386-ace3-e1c4d9afad76 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:26:14.858202 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a69db-1592-4386-ace3-e1c4d9afad76 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:26:14.858202 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a69db-1592-4386-ace3-e1c4d9afad76 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:26:14.859453 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 762/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 20:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:26:14.866232 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a3f23-4321-4860-bbe0-0058b5a2547f None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:26:14.866703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a3f23-4321-4860-bbe0-0058b5a2547f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:26:14.866809 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a3f23-4321-4860-bbe0-0058b5a2547f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:26:14.886751 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-7a3a3f23-4321-4860-bbe0-0058b5a2547f None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:26:14.887995 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7a3a3f23-4321-4860-bbe0-0058b5a2547f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:26:14.896205 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 761/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 20:26:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:26:15.050511 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5a69db-1592-4386-ace3-e1c4d9afad76 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:26:15.065184 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5a69db-1592-4386-ace3-e1c4d9afad76 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:26:15.070607 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad2295-0ed7-41d7-bd73-3883e2f17e22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:26:15.071194 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad2295-0ed7-41d7-bd73-3883e2f17e22 service nova] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:26:15.071472 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad2295-0ed7-41d7-bd73-3883e2f17e22 service nova] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:26:15.071743 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad2295-0ed7-41d7-bd73-3883e2f17e22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:26:15.080088 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cad2295-0ed7-41d7-bd73-3883e2f17e22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:26:15.081699 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cad2295-0ed7-41d7-bd73-3883e2f17e22 service nova] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:26:15.090822 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 763/1524] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:26:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:26:16.163551 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b0033-4250-4d41-be94-130154fa71b0 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:26:16.164072 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b0033-4250-4d41-be94-130154fa71b0 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:26:16.164072 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b0033-4250-4d41-be94-130154fa71b0 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:26:16.165467 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 762/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 20:26:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:26:16.171295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5dee5-03c9-4478-8f05-510de4bbdf19 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:26:16.171741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5dee5-03c9-4478-8f05-510de4bbdf19 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:26:16.171849 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5dee5-03c9-4478-8f05-510de4bbdf19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:26:16.191155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-86d5dee5-03c9-4478-8f05-510de4bbdf19 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:26:16.193163 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-86d5dee5-03c9-4478-8f05-510de4bbdf19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:26:16.200777 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 764/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 20:26:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:26:16.350806 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414b0033-4250-4d41-be94-130154fa71b0 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:26:16.365571 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414b0033-4250-4d41-be94-130154fa71b0 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:26:16.371169 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd32c053-8d6d-49da-bfc6-db9464642e08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:26:16.371758 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32c053-8d6d-49da-bfc6-db9464642e08 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:26:16.372044 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32c053-8d6d-49da-bfc6-db9464642e08 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:26:16.372314 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32c053-8d6d-49da-bfc6-db9464642e08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:26:16.380203 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd32c053-8d6d-49da-bfc6-db9464642e08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:26:16.381826 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd32c053-8d6d-49da-bfc6-db9464642e08 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:26:16.390988 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 763/1527] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:26:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:26:20.433224 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407ec7c-e0f9-4d5b-94b3-083af3e35fb6 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:26:20.433802 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407ec7c-e0f9-4d5b-94b3-083af3e35fb6 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:26:20.433802 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407ec7c-e0f9-4d5b-94b3-083af3e35fb6 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:26:20.435415 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 765/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 20:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:26:20.440515 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc813ce0-6656-4336-bd0a-cb7830f73ea5 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:26:20.440902 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc813ce0-6656-4336-bd0a-cb7830f73ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:26:20.441060 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc813ce0-6656-4336-bd0a-cb7830f73ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:26:20.462855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-bc813ce0-6656-4336-bd0a-cb7830f73ea5 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:26:20.464050 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-bc813ce0-6656-4336-bd0a-cb7830f73ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:26:20.471786 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 764/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 20:26:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:26:20.759808 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6407ec7c-e0f9-4d5b-94b3-083af3e35fb6 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:26:20.761517 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6407ec7c-e0f9-4d5b-94b3-083af3e35fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:26:20.841749 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6407ec7c-e0f9-4d5b-94b3-083af3e35fb6 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:26:20.845276 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0743fb-c5d3-4ed7-98cc-4beca2695785 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:26:20.845903 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0743fb-c5d3-4ed7-98cc-4beca2695785 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:26:20.846164 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0743fb-c5d3-4ed7-98cc-4beca2695785 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:26:20.846437 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0743fb-c5d3-4ed7-98cc-4beca2695785 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:26:20.855809 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0743fb-c5d3-4ed7-98cc-4beca2695785 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:26:20.857226 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0743fb-c5d3-4ed7-98cc-4beca2695785 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:26:20.866346 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 766/1530] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:26:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:26:23.015136 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc91f2-ad15-4572-944a-0505c3742e10 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:26:23.015709 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc91f2-ad15-4572-944a-0505c3742e10 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:26:23.015709 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc91f2-ad15-4572-944a-0505c3742e10 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:26:23.017019 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 765/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 20:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:26:23.022441 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c18bab-1833-466a-8aae-a215c8929752 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:26:23.022940 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c18bab-1833-466a-8aae-a215c8929752 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:26:23.023046 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c18bab-1833-466a-8aae-a215c8929752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:26:23.042449 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-97c18bab-1833-466a-8aae-a215c8929752 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:26:23.043647 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-97c18bab-1833-466a-8aae-a215c8929752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:26:23.052053 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 767/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 20:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:26:23.341719 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc91f2-ad15-4572-944a-0505c3742e10 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:26:23.343448 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-05fc91f2-ad15-4572-944a-0505c3742e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:26:23.351589 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=94 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:26:23.357957 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc91f2-ad15-4572-944a-0505c3742e10 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:26:23.361539 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f568cfa-17e8-4343-a4f2-9654063ac884 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:26:23.362131 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f568cfa-17e8-4343-a4f2-9654063ac884 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:26:23.362405 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f568cfa-17e8-4343-a4f2-9654063ac884 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:26:23.362676 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f568cfa-17e8-4343-a4f2-9654063ac884 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:26:23.372327 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f568cfa-17e8-4343-a4f2-9654063ac884 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:26:23.373763 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f568cfa-17e8-4343-a4f2-9654063ac884 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:26:23.382975 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 766/1533] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:26:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:26:27.160497 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1bea8-6624-4972-8fda-f9f539e0cdf2 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:26:27.161558 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1bea8-6624-4972-8fda-f9f539e0cdf2 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:26:27.161682 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1bea8-6624-4972-8fda-f9f539e0cdf2 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:26:27.162945 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 768/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 20:26:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:26:27.168035 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d431ce0-52eb-42e7-80e1-f04ce7fcc38e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:26:27.168439 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d431ce0-52eb-42e7-80e1-f04ce7fcc38e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:26:27.168587 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d431ce0-52eb-42e7-80e1-f04ce7fcc38e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:26:27.188181 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-1d431ce0-52eb-42e7-80e1-f04ce7fcc38e None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:26:27.189406 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1d431ce0-52eb-42e7-80e1-f04ce7fcc38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:26:27.197074 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 767/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 20:26:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:26:27.500724 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1bea8-6624-4972-8fda-f9f539e0cdf2 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:26:27.502395 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6dd1bea8-6624-4972-8fda-f9f539e0cdf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:26:27.579908 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1bea8-6624-4972-8fda-f9f539e0cdf2 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:26:27.583976 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474caa33-a9bd-4686-a335-d566c6ac98e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:26:27.584569 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474caa33-a9bd-4686-a335-d566c6ac98e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:26:27.584842 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474caa33-a9bd-4686-a335-d566c6ac98e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:26:27.585110 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474caa33-a9bd-4686-a335-d566c6ac98e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:26:27.593814 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474caa33-a9bd-4686-a335-d566c6ac98e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:26:27.595226 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474caa33-a9bd-4686-a335-d566c6ac98e7 service cinder] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:26:27.604912 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 769/1536] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:26:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:26:29.740537 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a795f-fd10-4dc4-be33-dbe633443728 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:26:29.741044 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a795f-fd10-4dc4-be33-dbe633443728 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:26:29.741174 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a795f-fd10-4dc4-be33-dbe633443728 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:26:29.742453 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 768/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 20:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:26:29.747961 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5681b-31ae-4387-bf26-2ffb878ce848 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:26:29.748388 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5681b-31ae-4387-bf26-2ffb878ce848 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:26:29.748500 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5681b-31ae-4387-bf26-2ffb878ce848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:26:29.768848 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-fba5681b-31ae-4387-bf26-2ffb878ce848 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:26:29.770128 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-fba5681b-31ae-4387-bf26-2ffb878ce848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:26:29.777785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 770/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 20:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:26:30.091237 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a795f-fd10-4dc4-be33-dbe633443728 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:26:30.092930 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-001a795f-fd10-4dc4-be33-dbe633443728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:26:30.107924 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a795f-fd10-4dc4-be33-dbe633443728 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:26:30.111408 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f82f5-b162-4363-a2da-c36d50066fcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:26:30.112007 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f82f5-b162-4363-a2da-c36d50066fcf service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:26:30.112281 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f82f5-b162-4363-a2da-c36d50066fcf service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:26:30.112546 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f82f5-b162-4363-a2da-c36d50066fcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:26:30.122904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f82f5-b162-4363-a2da-c36d50066fcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:26:30.124306 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f82f5-b162-4363-a2da-c36d50066fcf service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:26:30.135072 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 769/1539] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:26:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:26:39.763996 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=92 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:26:40.129821 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=14 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:27:56.303424 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83571982-67f2-45bf-a5e1-0ceebc57aa2a None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:56.304702 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83571982-67f2-45bf-a5e1-0ceebc57aa2a None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:56.305388 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83571982-67f2-45bf-a5e1-0ceebc57aa2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:56.339062 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-83571982-67f2-45bf-a5e1-0ceebc57aa2a None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:27:56.340459 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-83571982-67f2-45bf-a5e1-0ceebc57aa2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:27:56.350976 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 771/1540] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:27:56.360587 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8c8d7-0fe9-4dde-9c9f-5a5517c8dba4 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:56.361595 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8c8d7-0fe9-4dde-9c9f-5a5517c8dba4 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:56.362054 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8c8d7-0fe9-4dde-9c9f-5a5517c8dba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:56.391244 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-1eb8c8d7-0fe9-4dde-9c9f-5a5517c8dba4 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:27:56.392802 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1eb8c8d7-0fe9-4dde-9c9f-5a5517c8dba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:27:56.408977 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 770/1541] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:27:56.413428 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83571982-67f2-45bf-a5e1-0ceebc57aa2a None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:56.432249 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83571982-67f2-45bf-a5e1-0ceebc57aa2a None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:56.437089 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b313a3b-24c2-4a4b-8b86-49b701076a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:56.438289 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b313a3b-24c2-4a4b-8b86-49b701076a49 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:56.438567 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b313a3b-24c2-4a4b-8b86-49b701076a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:56.438837 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b313a3b-24c2-4a4b-8b86-49b701076a49 admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:56.439486 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b313a3b-24c2-4a4b-8b86-49b701076a49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:27:56.440045 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b313a3b-24c2-4a4b-8b86-49b701076a49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:56.441071 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: WARNING py.warnings [None req-3b313a3b-24c2-4a4b-8b86-49b701076a49 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 20:27:56.441071 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: warnings.warn(msg) Sep 09 20:27:56.441071 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: Sep 09 20:27:56.442119 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b313a3b-24c2-4a4b-8b86-49b701076a49 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:56.459012 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 772/1542] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:27:56.468031 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8c8d7-0fe9-4dde-9c9f-5a5517c8dba4 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:56.482824 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8c8d7-0fe9-4dde-9c9f-5a5517c8dba4 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:56.492980 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7a0b5-746d-4ec3-bc7a-36c897e2180d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:56.493980 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7a0b5-746d-4ec3-bc7a-36c897e2180d admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:56.494509 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7a0b5-746d-4ec3-bc7a-36c897e2180d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:56.494982 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7a0b5-746d-4ec3-bc7a-36c897e2180d admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:56.495840 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f7a0b5-746d-4ec3-bc7a-36c897e2180d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:56.497165 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: WARNING py.warnings [None req-b4f7a0b5-746d-4ec3-bc7a-36c897e2180d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 20:27:56.497165 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: warnings.warn(msg) Sep 09 20:27:56.497165 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: Sep 09 20:27:56.499152 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f7a0b5-746d-4ec3-bc7a-36c897e2180d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:56.568679 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 771/1543] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:27:56 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:27:56.573686 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b313a3b-24c2-4a4b-8b86-49b701076a49 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:56.595530 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b313a3b-24c2-4a4b-8b86-49b701076a49 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:56.663549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c7398-2d27-40de-8ab9-c16dc4d7beca None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:56.664116 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c7398-2d27-40de-8ab9-c16dc4d7beca None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:56.664599 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c7398-2d27-40de-8ab9-c16dc4d7beca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:56.687470 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3080a2-eba4-4337-b441-d93c53471884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:56.688153 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3080a2-eba4-4337-b441-d93c53471884 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:56.688433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3080a2-eba4-4337-b441-d93c53471884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:56.688708 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3080a2-eba4-4337-b441-d93c53471884 admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:56.689364 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3080a2-eba4-4337-b441-d93c53471884 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:56.690376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: WARNING py.warnings [None req-2b3080a2-eba4-4337-b441-d93c53471884 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 20:27:56.690376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: warnings.warn(msg) Sep 09 20:27:56.690376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: Sep 09 20:27:56.691584 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3080a2-eba4-4337-b441-d93c53471884 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:56.719720 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-990c7398-2d27-40de-8ab9-c16dc4d7beca None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:27:56.721438 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-990c7398-2d27-40de-8ab9-c16dc4d7beca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:27:56.729914 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 773/1544] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:27:56 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:27:56.733982 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3080a2-eba4-4337-b441-d93c53471884 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:56.740221 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 772/1545] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:27:56.745406 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3080a2-eba4-4337-b441-d93c53471884 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:56.749933 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025378b7-a288-4f69-9509-04be328a8107 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:56.750572 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025378b7-a288-4f69-9509-04be328a8107 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:56.751052 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025378b7-a288-4f69-9509-04be328a8107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:56.759579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611683da-e311-48e8-ae94-c5f544db2183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:56.760205 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611683da-e311-48e8-ae94-c5f544db2183 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:56.760483 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611683da-e311-48e8-ae94-c5f544db2183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:56.760755 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611683da-e311-48e8-ae94-c5f544db2183 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:56.764172 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611683da-e311-48e8-ae94-c5f544db2183 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:56.764172 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611683da-e311-48e8-ae94-c5f544db2183 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:56.781743 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 774/1546] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:27:56.786021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611683da-e311-48e8-ae94-c5f544db2183 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:56.786588 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-025378b7-a288-4f69-9509-04be328a8107 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:27:56.788047 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-025378b7-a288-4f69-9509-04be328a8107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:27:56.800615 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611683da-e311-48e8-ae94-c5f544db2183 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:56.813669 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 773/1547] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:27:56.814615 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef9c5f-f9b8-416b-9ec2-fa7fa5b34e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:56.815248 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef9c5f-f9b8-416b-9ec2-fa7fa5b34e58 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:56.815525 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef9c5f-f9b8-416b-9ec2-fa7fa5b34e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:56.815795 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef9c5f-f9b8-416b-9ec2-fa7fa5b34e58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:56.816172 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d470ad2-dece-40ee-91d7-f0a5fb9d9514 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:56.816665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d470ad2-dece-40ee-91d7-f0a5fb9d9514 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:56.817143 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d470ad2-dece-40ee-91d7-f0a5fb9d9514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:56.820912 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef9c5f-f9b8-416b-9ec2-fa7fa5b34e58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:56.822576 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef9c5f-f9b8-416b-9ec2-fa7fa5b34e58 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:56.841094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 775/1548] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:27:56.841992 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef9c5f-f9b8-416b-9ec2-fa7fa5b34e58 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:56.860730 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef9c5f-f9b8-416b-9ec2-fa7fa5b34e58 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:56.874458 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-2d470ad2-dece-40ee-91d7-f0a5fb9d9514 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:27:56.876765 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2d470ad2-dece-40ee-91d7-f0a5fb9d9514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:27:56.877645 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51462045-6aab-4d8c-b166-7c2f7ffbd75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:56.878506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51462045-6aab-4d8c-b166-7c2f7ffbd75c admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:56.879857 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51462045-6aab-4d8c-b166-7c2f7ffbd75c admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:56.880352 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51462045-6aab-4d8c-b166-7c2f7ffbd75c admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:56.881228 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51462045-6aab-4d8c-b166-7c2f7ffbd75c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:27:56.882671 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51462045-6aab-4d8c-b166-7c2f7ffbd75c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:56.884975 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51462045-6aab-4d8c-b166-7c2f7ffbd75c admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:56.901789 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 774/1549] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:27:56.902954 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d470ad2-dece-40ee-91d7-f0a5fb9d9514 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:56.921327 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 776/1550] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:27:56.926108 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41874e35-bf7c-4373-8c00-340d1c0b458e None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:56.928802 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d470ad2-dece-40ee-91d7-f0a5fb9d9514 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:56.929342 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41874e35-bf7c-4373-8c00-340d1c0b458e None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:56.929846 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41874e35-bf7c-4373-8c00-340d1c0b458e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:56.941812 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29b5f5-675a-4342-8392-170d20880471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:56.942844 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29b5f5-675a-4342-8392-170d20880471 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:56.943339 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29b5f5-675a-4342-8392-170d20880471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:56.943816 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29b5f5-675a-4342-8392-170d20880471 admin admin] PATH_INFO: `/v3/projects/b310c05b03394f11b63f0a28f13d67b6/users/95bc48b6541c4388b45435036a1c4525/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:56.958980 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-41874e35-bf7c-4373-8c00-340d1c0b458e None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:27:56.960186 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-41874e35-bf7c-4373-8c00-340d1c0b458e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:27:56.960324 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29b5f5-675a-4342-8392-170d20880471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b310c05b03394f11b63f0a28f13d67b6, user_id=95bc48b6541c4388b45435036a1c4525, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:56.961399 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: WARNING py.warnings [None req-3a29b5f5-675a-4342-8392-170d20880471 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 20:27:56.961399 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: warnings.warn(msg) Sep 09 20:27:56.961399 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: Sep 09 20:27:56.963344 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29b5f5-675a-4342-8392-170d20880471 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:56.968204 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 777/1551] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:27:56.969951 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31e83f-f83e-4b96-93e6-bfa57cb9e104 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:56.970331 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31e83f-f83e-4b96-93e6-bfa57cb9e104 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:56.970492 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31e83f-f83e-4b96-93e6-bfa57cb9e104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:56.978708 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 775/1552] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:27:56 2022] PUT /identity/v3/projects/b310c05b03394f11b63f0a28f13d67b6/users/95bc48b6541c4388b45435036a1c4525/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:27:56.979949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc062348-78e3-4aa8-b447-8bd9238b0dd6 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:56.980241 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc062348-78e3-4aa8-b447-8bd9238b0dd6 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:56.980518 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc062348-78e3-4aa8-b447-8bd9238b0dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:56.991056 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-bf31e83f-f83e-4b96-93e6-bfa57cb9e104 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:27:56.992534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-bf31e83f-f83e-4b96-93e6-bfa57cb9e104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:27:57.000174 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-bc062348-78e3-4aa8-b447-8bd9238b0dd6 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:27:57.001383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-bc062348-78e3-4aa8-b447-8bd9238b0dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:27:57.051595 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 778/1553] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:27:57.052834 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ac562-b9ba-4b5c-8a45-fe54e5d05923 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:57.053120 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ac562-b9ba-4b5c-8a45-fe54e5d05923 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:57.053437 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ac562-b9ba-4b5c-8a45-fe54e5d05923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:57.065452 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 776/1554] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:27:57.066472 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc062348-78e3-4aa8-b447-8bd9238b0dd6 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:57.073608 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-f24ac562-b9ba-4b5c-8a45-fe54e5d05923 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:27:57.074812 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-f24ac562-b9ba-4b5c-8a45-fe54e5d05923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:27:57.080943 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc062348-78e3-4aa8-b447-8bd9238b0dd6 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:57.082979 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 779/1555] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:27:57.085414 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1af82-1e4e-4297-8da3-d2e47afee703 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:57.085898 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1af82-1e4e-4297-8da3-d2e47afee703 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:57.086274 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1af82-1e4e-4297-8da3-d2e47afee703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:57.086411 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2fd95c-d60b-45d9-bc38-f2210dd81328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:57.086958 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2fd95c-d60b-45d9-bc38-f2210dd81328 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:57.087231 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2fd95c-d60b-45d9-bc38-f2210dd81328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:57.087555 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2fd95c-d60b-45d9-bc38-f2210dd81328 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:57.088143 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2fd95c-d60b-45d9-bc38-f2210dd81328 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:57.090198 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2fd95c-d60b-45d9-bc38-f2210dd81328 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:57.108790 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-a4f1af82-1e4e-4297-8da3-d2e47afee703 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:27:57.110664 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a4f1af82-1e4e-4297-8da3-d2e47afee703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:27:57.120498 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 780/1556] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:27:57.122040 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b137a3-f8ce-4af5-8a47-e5aa1d917967 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:57.122324 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b137a3-f8ce-4af5-8a47-e5aa1d917967 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:57.122598 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b137a3-f8ce-4af5-8a47-e5aa1d917967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:57.129409 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 777/1557] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:27:57 2022] POST /identity/v3/projects => generated 379 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:27:57.131111 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb949c-d2e2-49d0-997c-35e1fff21800 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:57.132348 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb949c-d2e2-49d0-997c-35e1fff21800 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:57.132640 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb949c-d2e2-49d0-997c-35e1fff21800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:57.143074 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-a5b137a3-f8ce-4af5-8a47-e5aa1d917967 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:27:57.144274 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a5b137a3-f8ce-4af5-8a47-e5aa1d917967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:27:57.151518 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-fecb949c-d2e2-49d0-997c-35e1fff21800 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:27:57.152697 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-fecb949c-d2e2-49d0-997c-35e1fff21800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:27:57.154217 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:27:57.204663 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:27:57.206685 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 781/1558] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:27:57.213364 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308d720-ddce-41f0-9aec-ce0f3fe14eb3 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:57.213364 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308d720-ddce-41f0-9aec-ce0f3fe14eb3 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:57.213364 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308d720-ddce-41f0-9aec-ce0f3fe14eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:57.221508 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 778/1559] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:27:57.222349 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb949c-d2e2-49d0-997c-35e1fff21800 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:57.228906 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-6308d720-ddce-41f0-9aec-ce0f3fe14eb3 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:27:57.230158 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6308d720-ddce-41f0-9aec-ce0f3fe14eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:27:57.231427 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb949c-d2e2-49d0-997c-35e1fff21800 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:57.236711 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b6afd-a139-4fcc-921f-b210703a40c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:57.237333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b6afd-a139-4fcc-921f-b210703a40c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:57.237614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b6afd-a139-4fcc-921f-b210703a40c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:57.237885 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b6afd-a139-4fcc-921f-b210703a40c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:57.239069 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8b6afd-a139-4fcc-921f-b210703a40c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:27:57.239069 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8b6afd-a139-4fcc-921f-b210703a40c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:57.239498 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 782/1560] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:27:57.240394 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8b6afd-a139-4fcc-921f-b210703a40c3 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:57.240551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec5ba4-0607-44c3-8549-e56785302f39 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:57.240831 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec5ba4-0607-44c3-8549-e56785302f39 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:57.241101 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec5ba4-0607-44c3-8549-e56785302f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:57.258905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 779/1561] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:27:57.258905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7e953-6854-4d68-b315-c3de1ee30d50 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:57.258905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7e953-6854-4d68-b315-c3de1ee30d50 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:57.258905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7e953-6854-4d68-b315-c3de1ee30d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:57.260628 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-ddec5ba4-0607-44c3-8549-e56785302f39 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:27:57.261849 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-ddec5ba4-0607-44c3-8549-e56785302f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:27:57.270232 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 783/1562] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:27:57.271399 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514bc1b0-e169-44b8-8df5-5396ff07dcb1 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:57.271685 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514bc1b0-e169-44b8-8df5-5396ff07dcb1 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:57.271961 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514bc1b0-e169-44b8-8df5-5396ff07dcb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:57.276098 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-74c7e953-6854-4d68-b315-c3de1ee30d50 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:27:57.277301 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-74c7e953-6854-4d68-b315-c3de1ee30d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:27:57.285788 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 780/1563] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:27:57.286449 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c7e953-6854-4d68-b315-c3de1ee30d50 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:57.294922 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-514bc1b0-e169-44b8-8df5-5396ff07dcb1 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:27:57.295641 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c7e953-6854-4d68-b315-c3de1ee30d50 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:57.296137 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-514bc1b0-e169-44b8-8df5-5396ff07dcb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:27:57.300395 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d9d87-fbdc-4b40-9a66-a12ddc38ffbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:57.300994 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d9d87-fbdc-4b40-9a66-a12ddc38ffbf admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:57.301301 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d9d87-fbdc-4b40-9a66-a12ddc38ffbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:57.301580 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d9d87-fbdc-4b40-9a66-a12ddc38ffbf admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:57.302207 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422d9d87-fbdc-4b40-9a66-a12ddc38ffbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:57.303973 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 784/1564] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:27:57.304433 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422d9d87-fbdc-4b40-9a66-a12ddc38ffbf admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:57.305189 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3829697-44cf-4c73-b5bd-b7fad82fe722 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:57.305505 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3829697-44cf-4c73-b5bd-b7fad82fe722 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:57.305785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3829697-44cf-4c73-b5bd-b7fad82fe722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:57.325147 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-e3829697-44cf-4c73-b5bd-b7fad82fe722 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:27:57.326371 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-e3829697-44cf-4c73-b5bd-b7fad82fe722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:27:57.327005 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 781/1565] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:27:57 2022] POST /identity/v3/users => generated 290 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:27:57.327754 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d9d87-fbdc-4b40-9a66-a12ddc38ffbf admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:57.337122 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 785/1566] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:27:57.337983 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3829697-44cf-4c73-b5bd-b7fad82fe722 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:57.341212 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d9d87-fbdc-4b40-9a66-a12ddc38ffbf admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:57.346606 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f58b3-dc53-4bae-aa4c-c599456308c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:57.347384 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f58b3-dc53-4bae-aa4c-c599456308c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:57.347664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f58b3-dc53-4bae-aa4c-c599456308c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:57.347937 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f58b3-dc53-4bae-aa4c-c599456308c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:57.348578 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f58b3-dc53-4bae-aa4c-c599456308c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:27:57.348885 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f58b3-dc53-4bae-aa4c-c599456308c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:57.350496 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f58b3-dc53-4bae-aa4c-c599456308c2 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:57.362647 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 782/1567] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:27:57.363258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f58b3-dc53-4bae-aa4c-c599456308c2 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:57.372993 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f58b3-dc53-4bae-aa4c-c599456308c2 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:57.378568 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68706dd0-3e7a-4dd3-8149-12c8466e4bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:57.379166 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68706dd0-3e7a-4dd3-8149-12c8466e4bca admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:57.379453 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68706dd0-3e7a-4dd3-8149-12c8466e4bca admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:57.379725 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68706dd0-3e7a-4dd3-8149-12c8466e4bca admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:57.380344 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68706dd0-3e7a-4dd3-8149-12c8466e4bca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:57.382312 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68706dd0-3e7a-4dd3-8149-12c8466e4bca admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:57.399908 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3829697-44cf-4c73-b5bd-b7fad82fe722 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:57.403736 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caceb257-d2e0-4592-93b5-f8fd3fb55f22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:57.404339 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caceb257-d2e0-4592-93b5-f8fd3fb55f22 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:57.404619 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caceb257-d2e0-4592-93b5-f8fd3fb55f22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:57.404890 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caceb257-d2e0-4592-93b5-f8fd3fb55f22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:57.413014 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 783/1568] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:27:57 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:27:57.413014 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68706dd0-3e7a-4dd3-8149-12c8466e4bca admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:57.414857 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caceb257-d2e0-4592-93b5-f8fd3fb55f22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:57.416297 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caceb257-d2e0-4592-93b5-f8fd3fb55f22 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:57.425823 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 786/1569] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:27:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:27:57.426607 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caceb257-d2e0-4592-93b5-f8fd3fb55f22 service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:57.435852 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caceb257-d2e0-4592-93b5-f8fd3fb55f22 service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:57.485124 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68706dd0-3e7a-4dd3-8149-12c8466e4bca admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:57.490359 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a543ca0-76d8-486c-8125-217bab8077fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:57.491075 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a543ca0-76d8-486c-8125-217bab8077fb admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:57.491721 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a543ca0-76d8-486c-8125-217bab8077fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:57.492109 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a543ca0-76d8-486c-8125-217bab8077fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:57.492883 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a543ca0-76d8-486c-8125-217bab8077fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:27:57.493180 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a543ca0-76d8-486c-8125-217bab8077fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:57.495107 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a543ca0-76d8-486c-8125-217bab8077fb admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:57.501015 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41982b5-c7de-477d-a4a8-147f4861cfc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:57.501891 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41982b5-c7de-477d-a4a8-147f4861cfc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:57.502176 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41982b5-c7de-477d-a4a8-147f4861cfc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:57.502456 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41982b5-c7de-477d-a4a8-147f4861cfc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:57.503097 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41982b5-c7de-477d-a4a8-147f4861cfc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:27:57.503408 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41982b5-c7de-477d-a4a8-147f4861cfc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:57.504991 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41982b5-c7de-477d-a4a8-147f4861cfc9 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:57.508477 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 784/1570] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:27:57.509520 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a543ca0-76d8-486c-8125-217bab8077fb admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:57.527656 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a543ca0-76d8-486c-8125-217bab8077fb admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:57.527888 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 787/1571] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:27:57.527888 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41982b5-c7de-477d-a4a8-147f4861cfc9 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:57.534155 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccfa2b-4ef8-40da-beb2-c6d1a58bfa4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:57.534785 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccfa2b-4ef8-40da-beb2-c6d1a58bfa4b admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:57.535116 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccfa2b-4ef8-40da-beb2-c6d1a58bfa4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:57.535395 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccfa2b-4ef8-40da-beb2-c6d1a58bfa4b admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:57.536070 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ccfa2b-4ef8-40da-beb2-c6d1a58bfa4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:27:57.536375 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ccfa2b-4ef8-40da-beb2-c6d1a58bfa4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:57.538035 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ccfa2b-4ef8-40da-beb2-c6d1a58bfa4b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:57.539578 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41982b5-c7de-477d-a4a8-147f4861cfc9 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:57.544490 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be936db-f208-4ade-9791-cb904979cda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:57.545105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be936db-f208-4ade-9791-cb904979cda7 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:57.545448 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be936db-f208-4ade-9791-cb904979cda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:57.545732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be936db-f208-4ade-9791-cb904979cda7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:57.546375 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be936db-f208-4ade-9791-cb904979cda7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:57.548031 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be936db-f208-4ade-9791-cb904979cda7 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:57.551949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 785/1572] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:27:57.552863 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccfa2b-4ef8-40da-beb2-c6d1a58bfa4b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:57.564310 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 788/1573] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:27:57.565156 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be936db-f208-4ade-9791-cb904979cda7 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:57.565498 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccfa2b-4ef8-40da-beb2-c6d1a58bfa4b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:57.571269 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86bddb5-821b-4964-89af-48916842f46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:57.572027 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86bddb5-821b-4964-89af-48916842f46f admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:57.572339 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86bddb5-821b-4964-89af-48916842f46f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:57.572817 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86bddb5-821b-4964-89af-48916842f46f admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:57.573486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86bddb5-821b-4964-89af-48916842f46f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:27:57.573817 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86bddb5-821b-4964-89af-48916842f46f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:57.575759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86bddb5-821b-4964-89af-48916842f46f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:57.576549 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be936db-f208-4ade-9791-cb904979cda7 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:57.581755 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ed006-fdd9-4c49-b544-b70bc69bce0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:57.582363 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ed006-fdd9-4c49-b544-b70bc69bce0c admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:57.582707 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ed006-fdd9-4c49-b544-b70bc69bce0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:57.582985 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ed006-fdd9-4c49-b544-b70bc69bce0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:57.583611 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ed006-fdd9-4c49-b544-b70bc69bce0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:57.585553 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ed006-fdd9-4c49-b544-b70bc69bce0c admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:57.588942 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 786/1574] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:27:57.590186 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86bddb5-821b-4964-89af-48916842f46f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:57.607780 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86bddb5-821b-4964-89af-48916842f46f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:57.609866 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:27:57.614297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37fc2a3-bfd2-4fb5-99e7-e50729eb4a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:57.615061 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37fc2a3-bfd2-4fb5-99e7-e50729eb4a26 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:57.615379 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37fc2a3-bfd2-4fb5-99e7-e50729eb4a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:57.615640 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37fc2a3-bfd2-4fb5-99e7-e50729eb4a26 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:57.616833 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37fc2a3-bfd2-4fb5-99e7-e50729eb4a26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:57.621923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37fc2a3-bfd2-4fb5-99e7-e50729eb4a26 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:57.635606 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 789/1575] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:27:57 2022] POST /identity/v3/projects => generated 365 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:27:57.635606 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ed006-fdd9-4c49-b544-b70bc69bce0c admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:57.654629 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 787/1576] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:27:57 2022] POST /identity/v3/users => generated 296 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:27:57.659085 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37fc2a3-bfd2-4fb5-99e7-e50729eb4a26 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:57.675594 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37fc2a3-bfd2-4fb5-99e7-e50729eb4a26 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:57.713605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:27:57.723965 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ed006-fdd9-4c49-b544-b70bc69bce0c admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:57.729205 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b9d2f-2b6e-4306-8477-f0763b4916d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:57.729847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b9d2f-2b6e-4306-8477-f0763b4916d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:57.730133 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b9d2f-2b6e-4306-8477-f0763b4916d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:57.730527 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b9d2f-2b6e-4306-8477-f0763b4916d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:57.731166 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443b9d2f-2b6e-4306-8477-f0763b4916d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:57.733083 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443b9d2f-2b6e-4306-8477-f0763b4916d6 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:57.740018 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db9690-087b-4209-9509-c17c1a363c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:57.740631 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db9690-087b-4209-9509-c17c1a363c9e admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:57.740914 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db9690-087b-4209-9509-c17c1a363c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:57.741194 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db9690-087b-4209-9509-c17c1a363c9e admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:57.742116 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db9690-087b-4209-9509-c17c1a363c9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:57.744010 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db9690-087b-4209-9509-c17c1a363c9e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:57.763616 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 790/1577] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:27:57 2022] POST /identity/v3/projects => generated 405 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:27:57.764534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b9d2f-2b6e-4306-8477-f0763b4916d6 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:57.781738 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 788/1578] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:27:57 2022] POST /identity/v3/projects => generated 383 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:27:57.783426 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db9690-087b-4209-9509-c17c1a363c9e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:57.841544 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b9d2f-2b6e-4306-8477-f0763b4916d6 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:57.883805 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db9690-087b-4209-9509-c17c1a363c9e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:57.890453 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3d9cc-0453-445b-ba31-c8fd4f8b25e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:57.892459 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3d9cc-0453-445b-ba31-c8fd4f8b25e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:57.892459 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3d9cc-0453-445b-ba31-c8fd4f8b25e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:57.892736 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3d9cc-0453-445b-ba31-c8fd4f8b25e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:57.893394 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d3d9cc-0453-445b-ba31-c8fd4f8b25e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:57.895005 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d3d9cc-0453-445b-ba31-c8fd4f8b25e6 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:57.907828 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 789/1579] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:27:57.908675 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3d9cc-0453-445b-ba31-c8fd4f8b25e6 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:57.919165 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b88e94-b0e7-47b1-b6b9-1c55f5f722e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:57.919864 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b88e94-b0e7-47b1-b6b9-1c55f5f722e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:57.920267 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b88e94-b0e7-47b1-b6b9-1c55f5f722e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:57.920535 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3d9cc-0453-445b-ba31-c8fd4f8b25e6 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:57.920982 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b88e94-b0e7-47b1-b6b9-1c55f5f722e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:57.921446 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b88e94-b0e7-47b1-b6b9-1c55f5f722e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:57.923653 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b88e94-b0e7-47b1-b6b9-1c55f5f722e5 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:57.926821 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4816ef-ab71-4c5c-b4a4-2cef2dc9b7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:57.927659 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4816ef-ab71-4c5c-b4a4-2cef2dc9b7d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:57.928279 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4816ef-ab71-4c5c-b4a4-2cef2dc9b7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:57.928279 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4816ef-ab71-4c5c-b4a4-2cef2dc9b7d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:57.929455 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4816ef-ab71-4c5c-b4a4-2cef2dc9b7d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:57.932299 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4816ef-ab71-4c5c-b4a4-2cef2dc9b7d4 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:57.955497 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 791/1580] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:27:57 2022] POST /identity/v3/projects => generated 387 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:27:57.956426 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b88e94-b0e7-47b1-b6b9-1c55f5f722e5 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:57.969319 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 790/1581] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:27:57 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:27:57.969319 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4816ef-ab71-4c5c-b4a4-2cef2dc9b7d4 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:57.974131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:27:58.029292 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b88e94-b0e7-47b1-b6b9-1c55f5f722e5 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:58.033934 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532c3e2-765f-4e86-a8b6-3d851d71db8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:58.034548 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532c3e2-765f-4e86-a8b6-3d851d71db8e admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:58.034822 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532c3e2-765f-4e86-a8b6-3d851d71db8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:58.035091 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532c3e2-765f-4e86-a8b6-3d851d71db8e admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:58.035713 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1532c3e2-765f-4e86-a8b6-3d851d71db8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:58.037659 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1532c3e2-765f-4e86-a8b6-3d851d71db8e admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:58.039048 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4816ef-ab71-4c5c-b4a4-2cef2dc9b7d4 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:58.045293 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9aaa35-3e01-4d89-970a-a864c8a42cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:58.046295 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9aaa35-3e01-4d89-970a-a864c8a42cf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:58.046690 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9aaa35-3e01-4d89-970a-a864c8a42cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:58.047521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9aaa35-3e01-4d89-970a-a864c8a42cf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:58.048177 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9aaa35-3e01-4d89-970a-a864c8a42cf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:58.050336 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9aaa35-3e01-4d89-970a-a864c8a42cf0 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:58.062017 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 792/1582] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:27:57 2022] POST /identity/v3/users => generated 283 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:27:58.066049 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 791/1583] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:27:58.066747 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532c3e2-765f-4e86-a8b6-3d851d71db8e admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:58.067540 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9aaa35-3e01-4d89-970a-a864c8a42cf0 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:58.078080 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532c3e2-765f-4e86-a8b6-3d851d71db8e admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:58.079123 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9aaa35-3e01-4d89-970a-a864c8a42cf0 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:58.083426 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8016520-0c02-49fe-a514-0ab0632d66e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:58.084025 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8016520-0c02-49fe-a514-0ab0632d66e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:58.084330 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8016520-0c02-49fe-a514-0ab0632d66e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:58.084792 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8016520-0c02-49fe-a514-0ab0632d66e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:58.085452 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3d2f5-70ef-4601-94fb-22c4c6489274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:58.085744 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8016520-0c02-49fe-a514-0ab0632d66e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:58.086239 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3d2f5-70ef-4601-94fb-22c4c6489274 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:58.086545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3d2f5-70ef-4601-94fb-22c4c6489274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:58.086955 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3d2f5-70ef-4601-94fb-22c4c6489274 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:58.087771 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af3d2f5-70ef-4601-94fb-22c4c6489274 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:58.088161 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8016520-0c02-49fe-a514-0ab0632d66e4 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:58.089733 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af3d2f5-70ef-4601-94fb-22c4c6489274 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:58.114562 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 793/1584] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:27:58 2022] POST /identity/v3/users => generated 303 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:27:58.115307 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8016520-0c02-49fe-a514-0ab0632d66e4 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:58.118462 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 792/1585] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:27:58 2022] POST /identity/v3/users => generated 292 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:27:58.119220 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3d2f5-70ef-4601-94fb-22c4c6489274 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:58.129559 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8016520-0c02-49fe-a514-0ab0632d66e4 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:58.138827 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a007c4c1-2799-4110-9ca4-7cadd61c3543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:58.139644 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007c4c1-2799-4110-9ca4-7cadd61c3543 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:58.139931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007c4c1-2799-4110-9ca4-7cadd61c3543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:58.140347 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007c4c1-2799-4110-9ca4-7cadd61c3543 admin admin] PATH_INFO: `/v3/projects/95899c2c318f459bb09bad337d79c1a2/users/fc41cb3dbe3845d794107f068c70db3f/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:58.140914 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3d2f5-70ef-4601-94fb-22c4c6489274 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:58.148170 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902a951-620b-4cee-bc14-e41886651ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:58.149005 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902a951-620b-4cee-bc14-e41886651ee3 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:58.149398 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902a951-620b-4cee-bc14-e41886651ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:58.149748 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902a951-620b-4cee-bc14-e41886651ee3 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:58.150486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8902a951-620b-4cee-bc14-e41886651ee3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:58.152563 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8902a951-620b-4cee-bc14-e41886651ee3 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:58.166918 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:27:58.171650 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a007c4c1-2799-4110-9ca4-7cadd61c3543 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95899c2c318f459bb09bad337d79c1a2, user_id=fc41cb3dbe3845d794107f068c70db3f, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:58.175877 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a007c4c1-2799-4110-9ca4-7cadd61c3543 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:58.185625 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 793/1586] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:27:58 2022] POST /identity/v3/users => generated 294 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:27:58.187006 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902a951-620b-4cee-bc14-e41886651ee3 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:58.195882 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 794/1587] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:27:58 2022] PUT /identity/v3/projects/95899c2c318f459bb09bad337d79c1a2/users/fc41cb3dbe3845d794107f068c70db3f/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:27:58.196814 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a007c4c1-2799-4110-9ca4-7cadd61c3543 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:58.202561 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902a951-620b-4cee-bc14-e41886651ee3 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:58.281445 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44e273-36d9-482a-917f-ab014ee38ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:58.282120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44e273-36d9-482a-917f-ab014ee38ad7 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:58.282773 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44e273-36d9-482a-917f-ab014ee38ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:58.283137 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44e273-36d9-482a-917f-ab014ee38ad7 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:58.284012 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec44e273-36d9-482a-917f-ab014ee38ad7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:58.286116 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec44e273-36d9-482a-917f-ab014ee38ad7 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:58.287755 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a007c4c1-2799-4110-9ca4-7cadd61c3543 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:58.293295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38e197-ca7c-4c2f-9049-567fb8723cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:58.294224 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38e197-ca7c-4c2f-9049-567fb8723cd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:58.294992 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38e197-ca7c-4c2f-9049-567fb8723cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:58.295522 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38e197-ca7c-4c2f-9049-567fb8723cd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:58.298153 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38e197-ca7c-4c2f-9049-567fb8723cd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:58.301644 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38e197-ca7c-4c2f-9049-567fb8723cd2 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:58.311073 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 794/1588] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:27:58 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:27:58.311979 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44e273-36d9-482a-917f-ab014ee38ad7 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:58.322957 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44e273-36d9-482a-917f-ab014ee38ad7 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:58.327582 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5000e0-5817-445a-83c5-80f58b03ec4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:58.328247 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5000e0-5817-445a-83c5-80f58b03ec4a admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:58.328524 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5000e0-5817-445a-83c5-80f58b03ec4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:58.328794 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5000e0-5817-445a-83c5-80f58b03ec4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:58.329465 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5000e0-5817-445a-83c5-80f58b03ec4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:58.331045 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5000e0-5817-445a-83c5-80f58b03ec4a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:58.332309 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 795/1589] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:27:58.336310 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38e197-ca7c-4c2f-9049-567fb8723cd2 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:58.348819 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 795/1590] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:27:58.349726 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5000e0-5817-445a-83c5-80f58b03ec4a admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:58.350821 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38e197-ca7c-4c2f-9049-567fb8723cd2 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:58.356756 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a630dc-5e3b-417f-8d9d-ed387fc8a4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:58.357937 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a630dc-5e3b-417f-8d9d-ed387fc8a4ff admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:58.358332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a630dc-5e3b-417f-8d9d-ed387fc8a4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:58.358813 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a630dc-5e3b-417f-8d9d-ed387fc8a4ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:58.360293 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a630dc-5e3b-417f-8d9d-ed387fc8a4ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:58.362075 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a630dc-5e3b-417f-8d9d-ed387fc8a4ff admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:58.364074 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5000e0-5817-445a-83c5-80f58b03ec4a admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:58.371020 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357156f0-d665-42de-bc79-bea575a6da6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:58.371689 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357156f0-d665-42de-bc79-bea575a6da6b admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:58.371968 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357156f0-d665-42de-bc79-bea575a6da6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:58.372240 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357156f0-d665-42de-bc79-bea575a6da6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:58.372937 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357156f0-d665-42de-bc79-bea575a6da6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:58.374604 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357156f0-d665-42de-bc79-bea575a6da6b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:58.375929 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 796/1591] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:27:58.376897 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a630dc-5e3b-417f-8d9d-ed387fc8a4ff admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:58.390195 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a630dc-5e3b-417f-8d9d-ed387fc8a4ff admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:58.393839 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 796/1592] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:27:58.395391 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5416bb-5fbd-4ff1-93d0-8d333f8186a1 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:58.395900 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5416bb-5fbd-4ff1-93d0-8d333f8186a1 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:58.396449 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5416bb-5fbd-4ff1-93d0-8d333f8186a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:58.401194 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4dcea-7f8b-4ee9-9c05-e332a3d0b596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:58.402007 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4dcea-7f8b-4ee9-9c05-e332a3d0b596 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:58.403300 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4dcea-7f8b-4ee9-9c05-e332a3d0b596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:58.403588 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4dcea-7f8b-4ee9-9c05-e332a3d0b596 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:58.404292 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e4dcea-7f8b-4ee9-9c05-e332a3d0b596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:58.407140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e4dcea-7f8b-4ee9-9c05-e332a3d0b596 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:58.423279 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 797/1593] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:27:58.423279 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4dcea-7f8b-4ee9-9c05-e332a3d0b596 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:58.425278 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-5f5416bb-5fbd-4ff1-93d0-8d333f8186a1 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:27:58.426738 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5f5416bb-5fbd-4ff1-93d0-8d333f8186a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:27:58.433681 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4dcea-7f8b-4ee9-9c05-e332a3d0b596 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:58.439220 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36860aa2-d71c-4c65-9c9d-d6181cb43303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:58.440053 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 797/1594] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:27:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:27:58.440403 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36860aa2-d71c-4c65-9c9d-d6181cb43303 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:58.440691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36860aa2-d71c-4c65-9c9d-d6181cb43303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:58.441076 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36860aa2-d71c-4c65-9c9d-d6181cb43303 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:58.441275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5416bb-5fbd-4ff1-93d0-8d333f8186a1 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:58.441753 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36860aa2-d71c-4c65-9c9d-d6181cb43303 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:58.449645 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36860aa2-d71c-4c65-9c9d-d6181cb43303 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:58.456106 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5416bb-5fbd-4ff1-93d0-8d333f8186a1 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:58.461372 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b56bd39-a519-4fdb-94c8-850f37ce72d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:58.461583 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 798/1595] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:27:58.462002 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56bd39-a519-4fdb-94c8-850f37ce72d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:58.462265 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56bd39-a519-4fdb-94c8-850f37ce72d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:58.462376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36860aa2-d71c-4c65-9c9d-d6181cb43303 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:58.462550 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56bd39-a519-4fdb-94c8-850f37ce72d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:58.463270 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b56bd39-a519-4fdb-94c8-850f37ce72d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:58.465221 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b56bd39-a519-4fdb-94c8-850f37ce72d5 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:58.474225 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36860aa2-d71c-4c65-9c9d-d6181cb43303 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:58.480170 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 798/1596] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:27:58.480316 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b15ca8-a3d4-4a11-857a-1454f067f387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:58.481098 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b15ca8-a3d4-4a11-857a-1454f067f387 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:58.481263 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b56bd39-a519-4fdb-94c8-850f37ce72d5 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:58.481441 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b15ca8-a3d4-4a11-857a-1454f067f387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:58.481823 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b15ca8-a3d4-4a11-857a-1454f067f387 admin admin] PATH_INFO: `/v3/projects/f07defd609684351b4b7d0d110212a7c/users/a95746e7a84541199b43138b1a348386/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:58.493713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b56bd39-a519-4fdb-94c8-850f37ce72d5 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:58.498756 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b15ca8-a3d4-4a11-857a-1454f067f387 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f07defd609684351b4b7d0d110212a7c, user_id=a95746e7a84541199b43138b1a348386, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:58.499360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f1adb2-d429-4a2a-b751-873a64bf74a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:58.500672 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1adb2-d429-4a2a-b751-873a64bf74a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:58.500958 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1adb2-d429-4a2a-b751-873a64bf74a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:58.501231 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1adb2-d429-4a2a-b751-873a64bf74a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:58.501945 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f1adb2-d429-4a2a-b751-873a64bf74a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:58.502908 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b15ca8-a3d4-4a11-857a-1454f067f387 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:58.503768 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f1adb2-d429-4a2a-b751-873a64bf74a6 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:58.513991 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:27:58.517284 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 799/1597] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:27:58.518120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f1adb2-d429-4a2a-b751-873a64bf74a6 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:58.526434 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 799/1598] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:27:58 2022] PUT /identity/v3/projects/f07defd609684351b4b7d0d110212a7c/users/a95746e7a84541199b43138b1a348386/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:27:58.527592 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b15ca8-a3d4-4a11-857a-1454f067f387 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:58.533870 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f1adb2-d429-4a2a-b751-873a64bf74a6 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:58.607743 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b15ca8-a3d4-4a11-857a-1454f067f387 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:58.609190 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b9e57-1de9-4ecb-a19e-6a13c6d5314e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:58.610345 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b9e57-1de9-4ecb-a19e-6a13c6d5314e admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:58.610637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b9e57-1de9-4ecb-a19e-6a13c6d5314e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:58.610960 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b9e57-1de9-4ecb-a19e-6a13c6d5314e admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:58.611612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b9e57-1de9-4ecb-a19e-6a13c6d5314e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:58.613335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a292f3e9-f081-4122-a67c-92ad39725065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:58.614019 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292f3e9-f081-4122-a67c-92ad39725065 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:58.614248 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b9e57-1de9-4ecb-a19e-6a13c6d5314e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:58.614423 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292f3e9-f081-4122-a67c-92ad39725065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:58.614788 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292f3e9-f081-4122-a67c-92ad39725065 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:58.615515 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a292f3e9-f081-4122-a67c-92ad39725065 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:58.617779 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a292f3e9-f081-4122-a67c-92ad39725065 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:58.627399 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 800/1599] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:27:58.628321 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b9e57-1de9-4ecb-a19e-6a13c6d5314e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:58.633393 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 800/1600] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:27:58.634383 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a292f3e9-f081-4122-a67c-92ad39725065 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:58.648635 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a292f3e9-f081-4122-a67c-92ad39725065 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:58.654183 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961e8f3-2b9f-414d-9af7-548d844a808b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:58.658580 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961e8f3-2b9f-414d-9af7-548d844a808b admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:58.658580 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961e8f3-2b9f-414d-9af7-548d844a808b admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:58.658580 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961e8f3-2b9f-414d-9af7-548d844a808b admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:58.658580 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961e8f3-2b9f-414d-9af7-548d844a808b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:58.659430 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961e8f3-2b9f-414d-9af7-548d844a808b admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:58.680201 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 801/1601] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:27:58.681195 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961e8f3-2b9f-414d-9af7-548d844a808b admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:58.685942 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:27:58.696031 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961e8f3-2b9f-414d-9af7-548d844a808b admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:58.702037 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a090c5-7073-4cde-93af-e12de85b3000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:58.702860 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a090c5-7073-4cde-93af-e12de85b3000 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:58.703910 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a090c5-7073-4cde-93af-e12de85b3000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:58.704196 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a090c5-7073-4cde-93af-e12de85b3000 admin admin] PATH_INFO: `/v3/projects/afe2f4380cb24459929acfd48d2c40ad/users/9d15eb60e90f44aab02044a61e7aa461/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:58.722316 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a090c5-7073-4cde-93af-e12de85b3000 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afe2f4380cb24459929acfd48d2c40ad, user_id=9d15eb60e90f44aab02044a61e7aa461, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:58.724823 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b9e57-1de9-4ecb-a19e-6a13c6d5314e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:58.725127 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a090c5-7073-4cde-93af-e12de85b3000 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:58.730310 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a21190-cbb8-4801-9572-4731864363da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:58.731177 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a21190-cbb8-4801-9572-4731864363da service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:58.731693 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a21190-cbb8-4801-9572-4731864363da service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:58.732118 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a21190-cbb8-4801-9572-4731864363da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:58.754461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 802/1602] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:27:58 2022] PUT /identity/v3/projects/afe2f4380cb24459929acfd48d2c40ad/users/9d15eb60e90f44aab02044a61e7aa461/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:27:58.757868 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a21190-cbb8-4801-9572-4731864363da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:58.757966 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a090c5-7073-4cde-93af-e12de85b3000 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:58.758051 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a21190-cbb8-4801-9572-4731864363da service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:58.758051 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 801/1603] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:27:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:27:58.758341 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036eb61-8c51-4607-b6eb-23e1980ac189 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:58.758627 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036eb61-8c51-4607-b6eb-23e1980ac189 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:58.758902 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036eb61-8c51-4607-b6eb-23e1980ac189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:58.779085 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f036eb61-8c51-4607-b6eb-23e1980ac189 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:27:58.780320 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f036eb61-8c51-4607-b6eb-23e1980ac189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:27:58.871217 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 802/1604] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:27:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:27:58.872790 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f036eb61-8c51-4607-b6eb-23e1980ac189 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:58.888703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f036eb61-8c51-4607-b6eb-23e1980ac189 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:58.891563 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a090c5-7073-4cde-93af-e12de85b3000 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:58.894780 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58056a-830b-4345-ab43-e7957bf67dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:58.895767 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58056a-830b-4345-ab43-e7957bf67dcc admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:58.896165 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58056a-830b-4345-ab43-e7957bf67dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:58.896594 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58056a-830b-4345-ab43-e7957bf67dcc admin admin] PATH_INFO: `/v3/projects/a559fc57f6444f3088f93ed92fb8afc7/users/5cfb93f9864f435ba544dbb08be07b1a/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:58.904981 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b93e2d-35af-45cc-a6c5-d158dc13cf63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:58.905699 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b93e2d-35af-45cc-a6c5-d158dc13cf63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:58.905983 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b93e2d-35af-45cc-a6c5-d158dc13cf63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:58.906296 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b93e2d-35af-45cc-a6c5-d158dc13cf63 admin admin] PATH_INFO: `/v3/projects/a8041d5ff2ad45fc8e7577b624585332/users/8b4fc11c58544a34b31de46351b5e91e/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:58.923933 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58056a-830b-4345-ab43-e7957bf67dcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a559fc57f6444f3088f93ed92fb8afc7, user_id=5cfb93f9864f435ba544dbb08be07b1a, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:58.927852 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58056a-830b-4345-ab43-e7957bf67dcc admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:58.929540 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b93e2d-35af-45cc-a6c5-d158dc13cf63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8041d5ff2ad45fc8e7577b624585332, user_id=8b4fc11c58544a34b31de46351b5e91e, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:58.932415 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b93e2d-35af-45cc-a6c5-d158dc13cf63 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:58.948004 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 803/1605] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:27:58 2022] PUT /identity/v3/projects/a559fc57f6444f3088f93ed92fb8afc7/users/5cfb93f9864f435ba544dbb08be07b1a/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:27:58.949799 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58056a-830b-4345-ab43-e7957bf67dcc admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:58.979677 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 803/1606] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:27:58 2022] PUT /identity/v3/projects/a8041d5ff2ad45fc8e7577b624585332/users/8b4fc11c58544a34b31de46351b5e91e/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:27:58.980708 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b93e2d-35af-45cc-a6c5-d158dc13cf63 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:59.047631 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:27:59.065020 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58056a-830b-4345-ab43-e7957bf67dcc admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:59.071826 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15e73a-09bb-4ce7-b794-9060431a86ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:59.072053 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b93e2d-35af-45cc-a6c5-d158dc13cf63 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:59.072567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15e73a-09bb-4ce7-b794-9060431a86ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:59.073167 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15e73a-09bb-4ce7-b794-9060431a86ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:59.073526 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15e73a-09bb-4ce7-b794-9060431a86ea admin admin] PATH_INFO: `/v3/projects/8c1c909559e44322a07f7e4a463424a5/users/db1f365ebf794941b3999adb37c48100/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:59.078909 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcba0c0-2080-43fe-9f02-685d9a36ee9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:59.079988 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcba0c0-2080-43fe-9f02-685d9a36ee9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:59.080526 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcba0c0-2080-43fe-9f02-685d9a36ee9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:59.081018 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcba0c0-2080-43fe-9f02-685d9a36ee9f admin admin] PATH_INFO: `/v3/projects/3bae95c402e44346959aca56968761aa/users/eb174dab2ed04d34857dcd27a17480f9/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:59.103091 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcba0c0-2080-43fe-9f02-685d9a36ee9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bae95c402e44346959aca56968761aa, user_id=eb174dab2ed04d34857dcd27a17480f9, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:59.104331 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15e73a-09bb-4ce7-b794-9060431a86ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c1c909559e44322a07f7e4a463424a5, user_id=db1f365ebf794941b3999adb37c48100, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:59.107338 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15e73a-09bb-4ce7-b794-9060431a86ea admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:59.112442 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcba0c0-2080-43fe-9f02-685d9a36ee9f admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:59.129673 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 804/1607] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:27:58 2022] PUT /identity/v3/projects/8c1c909559e44322a07f7e4a463424a5/users/db1f365ebf794941b3999adb37c48100/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:27:59.129673 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90017992-1281-413b-a436-212c8c059bcb None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:59.135454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90017992-1281-413b-a436-212c8c059bcb None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:59.135796 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90017992-1281-413b-a436-212c8c059bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:59.147309 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 804/1608] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:27:58 2022] PUT /identity/v3/projects/3bae95c402e44346959aca56968761aa/users/eb174dab2ed04d34857dcd27a17480f9/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:27:59.148085 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcba0c0-2080-43fe-9f02-685d9a36ee9f admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:59.192167 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-90017992-1281-413b-a436-212c8c059bcb None None] MFA Rules not processed for user `9d15eb60e90f44aab02044a61e7aa461`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:27:59.193469 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-90017992-1281-413b-a436-212c8c059bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:27:59.220027 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:27:59.238110 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcba0c0-2080-43fe-9f02-685d9a36ee9f admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:59.244902 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5053fb-c67f-4e7a-8bbd-280bf836fd4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:59.245095 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5053fb-c67f-4e7a-8bbd-280bf836fd4f service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:59.245761 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5053fb-c67f-4e7a-8bbd-280bf836fd4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:59.246130 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5053fb-c67f-4e7a-8bbd-280bf836fd4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:59.287241 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 805/1609] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:27:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:27:59.288415 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f5901-afd8-4186-91cb-6cd8eeebebc4 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:59.288752 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f5901-afd8-4186-91cb-6cd8eeebebc4 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:59.289076 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f5901-afd8-4186-91cb-6cd8eeebebc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:59.387448 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-630f5901-afd8-4186-91cb-6cd8eeebebc4 None None] MFA Rules not processed for user `5cfb93f9864f435ba544dbb08be07b1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:27:59.388693 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-630f5901-afd8-4186-91cb-6cd8eeebebc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:27:59.441619 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5053fb-c67f-4e7a-8bbd-280bf836fd4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:59.441619 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5053fb-c67f-4e7a-8bbd-280bf836fd4f service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:59.468780 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 805/1610] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:27:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:27:59.470264 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb0ea0-4f1b-4c9d-a83d-e8ab8fa0b61e None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:59.470552 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb0ea0-4f1b-4c9d-a83d-e8ab8fa0b61e None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:59.470830 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb0ea0-4f1b-4c9d-a83d-e8ab8fa0b61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:59.527464 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-16fb0ea0-4f1b-4c9d-a83d-e8ab8fa0b61e None None] MFA Rules not processed for user `8b4fc11c58544a34b31de46351b5e91e`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:27:59.529447 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-16fb0ea0-4f1b-4c9d-a83d-e8ab8fa0b61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:27:59.532356 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 806/1611] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:27:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:27:59.533411 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c74ea-729b-4bfb-8d91-b5e7523256c0 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:59.533702 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c74ea-729b-4bfb-8d91-b5e7523256c0 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:59.533984 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c74ea-729b-4bfb-8d91-b5e7523256c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:59.574334 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-b11c74ea-729b-4bfb-8d91-b5e7523256c0 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:27:59.575590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b11c74ea-729b-4bfb-8d91-b5e7523256c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:27:59.585996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 807/1612] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:27:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:27:59.587737 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36992b68-6073-4091-8507-a8d6f2b71326 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:59.588017 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36992b68-6073-4091-8507-a8d6f2b71326 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:59.588294 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36992b68-6073-4091-8507-a8d6f2b71326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:59.619033 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 806/1613] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:27:59 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:27:59.620540 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb0ea0-4f1b-4c9d-a83d-e8ab8fa0b61e None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:59.630037 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:27:59.635529 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-36992b68-6073-4091-8507-a8d6f2b71326 None None] MFA Rules not processed for user `eb174dab2ed04d34857dcd27a17480f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:27:59.637080 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-36992b68-6073-4091-8507-a8d6f2b71326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:27:59.713359 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb0ea0-4f1b-4c9d-a83d-e8ab8fa0b61e None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:59.718621 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1636c4c-37b3-42d3-aa65-7bfbbbeea493 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:59.719967 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1636c4c-37b3-42d3-aa65-7bfbbbeea493 service cinder] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:59.720318 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1636c4c-37b3-42d3-aa65-7bfbbbeea493 service cinder] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:59.720602 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1636c4c-37b3-42d3-aa65-7bfbbbeea493 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:59.733065 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1636c4c-37b3-42d3-aa65-7bfbbbeea493 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:59.736513 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1636c4c-37b3-42d3-aa65-7bfbbbeea493 service cinder] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:59.753292 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 807/1615] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:27:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:27:59.753292 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1636c4c-37b3-42d3-aa65-7bfbbbeea493 service cinder] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:59.762639 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 808/1615] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:27:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:27:59.763817 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36992b68-6073-4091-8507-a8d6f2b71326 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:59.766265 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1636c4c-37b3-42d3-aa65-7bfbbbeea493 service cinder] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:59.772998 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d0408-59c3-4cb4-ac23-c85ea8a178f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:59.774179 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d0408-59c3-4cb4-ac23-c85ea8a178f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:59.774786 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d0408-59c3-4cb4-ac23-c85ea8a178f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:59.775233 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d0408-59c3-4cb4-ac23-c85ea8a178f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:59.776701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369d0408-59c3-4cb4-ac23-c85ea8a178f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:59.780190 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369d0408-59c3-4cb4-ac23-c85ea8a178f1 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:59.789578 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36992b68-6073-4091-8507-a8d6f2b71326 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:59.805174 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf2963-12df-4a32-9810-e56244af1e3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:59.806872 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf2963-12df-4a32-9810-e56244af1e3a service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:59.807277 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf2963-12df-4a32-9810-e56244af1e3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:59.808818 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf2963-12df-4a32-9810-e56244af1e3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:59.825330 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 808/1616] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:27:59 2022] POST /identity/v3/projects => generated 385 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:27:59.827344 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d0408-59c3-4cb4-ac23-c85ea8a178f1 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:59.828894 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf2963-12df-4a32-9810-e56244af1e3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:59.830967 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf2963-12df-4a32-9810-e56244af1e3a service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:59.859041 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 809/1617] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:27:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:27:59.865239 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23a93b-de8b-4840-91ee-6ea602bd768b None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:59.865545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23a93b-de8b-4840-91ee-6ea602bd768b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:59.865814 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23a93b-de8b-4840-91ee-6ea602bd768b None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:59.867427 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 810/1618] 10.222.0.26 () {54 vars in 962 bytes} [Fri Sep 9 20:27:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:27:59.872244 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23a93b-de8b-4840-91ee-6ea602bd768b None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:27:59.923456 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d0408-59c3-4cb4-ac23-c85ea8a178f1 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:27:59.929802 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0da5cc0-6162-41de-9c51-4ffdfbcbedce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:27:59.930905 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da5cc0-6162-41de-9c51-4ffdfbcbedce admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:59.931416 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da5cc0-6162-41de-9c51-4ffdfbcbedce admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:59.931790 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da5cc0-6162-41de-9c51-4ffdfbcbedce admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:27:59.932529 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0da5cc0-6162-41de-9c51-4ffdfbcbedce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:27:59.934859 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0da5cc0-6162-41de-9c51-4ffdfbcbedce admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:27:59.941432 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:27:59.990416 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 809/1619] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:27:59 2022] POST /identity/v3/projects => generated 405 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:27:59.993374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723a2f0-cc33-4833-864d-303d5096e21c None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:27:59.995083 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723a2f0-cc33-4833-864d-303d5096e21c None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:27:59.995658 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723a2f0-cc33-4833-864d-303d5096e21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:00.072011 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23a93b-de8b-4840-91ee-6ea602bd768b None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:00.113210 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-6723a2f0-cc33-4833-864d-303d5096e21c None None] MFA Rules not processed for user `87ece6128f9245f7b49f3e7a9faaed22`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:00.117300 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6723a2f0-cc33-4833-864d-303d5096e21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:00.235300 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7301f5-2d3e-499b-b4b5-f337acdc35c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:00.236038 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7301f5-2d3e-499b-b4b5-f337acdc35c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:00.236895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7301f5-2d3e-499b-b4b5-f337acdc35c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:00.238478 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7301f5-2d3e-499b-b4b5-f337acdc35c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:00.239917 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7301f5-2d3e-499b-b4b5-f337acdc35c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:00.245050 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7301f5-2d3e-499b-b4b5-f337acdc35c3 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:00.256000 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:00.301889 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 810/1620] 10.222.0.26 () {58 vars in 1059 bytes} [Fri Sep 9 20:27:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:00.302459 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 811/1621] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:27:59 2022] POST /identity/v3/users => generated 299 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:00.304508 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7301f5-2d3e-499b-b4b5-f337acdc35c3 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:00.306467 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723a2f0-cc33-4833-864d-303d5096e21c None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:00.330985 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7301f5-2d3e-499b-b4b5-f337acdc35c3 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:00.339391 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723a2f0-cc33-4833-864d-303d5096e21c None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:00.363380 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d926377b-ee23-411d-b884-ea788b13e778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:00.365994 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6bf65-3284-4b1f-a6f4-1f2f77b32a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:00.366886 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6bf65-3284-4b1f-a6f4-1f2f77b32a96 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:00.367167 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6bf65-3284-4b1f-a6f4-1f2f77b32a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:00.367442 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6bf65-3284-4b1f-a6f4-1f2f77b32a96 admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:00.367903 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d926377b-ee23-411d-b884-ea788b13e778 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:00.368264 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d926377b-ee23-411d-b884-ea788b13e778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:00.368751 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d926377b-ee23-411d-b884-ea788b13e778 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:00.370702 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d926377b-ee23-411d-b884-ea788b13e778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:00.372884 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d926377b-ee23-411d-b884-ea788b13e778 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:00.374739 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc6bf65-3284-4b1f-a6f4-1f2f77b32a96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:00.380883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc6bf65-3284-4b1f-a6f4-1f2f77b32a96 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:00.418399 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 812/1622] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:00.419339 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d926377b-ee23-411d-b884-ea788b13e778 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:00.448271 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 811/1623] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:28:00 2022] POST /identity/v3/users => generated 309 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:00.453963 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6bf65-3284-4b1f-a6f4-1f2f77b32a96 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:00.460362 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d926377b-ee23-411d-b884-ea788b13e778 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:00.483798 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ad879-43a8-4e98-a8b8-aabec042fbfa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:00.486994 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ad879-43a8-4e98-a8b8-aabec042fbfa service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:00.488099 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ad879-43a8-4e98-a8b8-aabec042fbfa service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:00.488526 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ad879-43a8-4e98-a8b8-aabec042fbfa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:00.492416 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6bf65-3284-4b1f-a6f4-1f2f77b32a96 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:00.509957 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb031c-0ce7-4e32-aec1-8a393ebd0b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:00.514104 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb031c-0ce7-4e32-aec1-8a393ebd0b28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:00.516308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb031c-0ce7-4e32-aec1-8a393ebd0b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:00.516613 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb031c-0ce7-4e32-aec1-8a393ebd0b28 admin admin] PATH_INFO: `/v3/projects/314d8d99f97549b0ab7f54664473b053/users/4f68963381b04ffea13a17b31af692a1/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:00.557197 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bb031c-0ce7-4e32-aec1-8a393ebd0b28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=314d8d99f97549b0ab7f54664473b053, user_id=4f68963381b04ffea13a17b31af692a1, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:00.567313 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bb031c-0ce7-4e32-aec1-8a393ebd0b28 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:00.602549 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 812/1624] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:00 2022] PUT /identity/v3/projects/314d8d99f97549b0ab7f54664473b053/users/4f68963381b04ffea13a17b31af692a1/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:00.608831 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb031c-0ce7-4e32-aec1-8a393ebd0b28 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:00.648057 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ad879-43a8-4e98-a8b8-aabec042fbfa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:00.650262 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ad879-43a8-4e98-a8b8-aabec042fbfa service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:00.685491 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 813/1625] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:00.686484 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ad879-43a8-4e98-a8b8-aabec042fbfa service nova] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:00.711925 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ad879-43a8-4e98-a8b8-aabec042fbfa service nova] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:00.764475 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb031c-0ce7-4e32-aec1-8a393ebd0b28 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:00.775648 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9274567-1ac7-465b-af01-71090d83286e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:00.776832 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9274567-1ac7-465b-af01-71090d83286e admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:00.776832 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9274567-1ac7-465b-af01-71090d83286e admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:00.777201 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9274567-1ac7-465b-af01-71090d83286e admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:00.777686 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9274567-1ac7-465b-af01-71090d83286e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:00.779819 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9274567-1ac7-465b-af01-71090d83286e admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:00.820673 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 813/1626] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:00.823330 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9274567-1ac7-465b-af01-71090d83286e admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:00.850966 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:00.854242 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9274567-1ac7-465b-af01-71090d83286e admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:00.865333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a1007-4ca4-48af-9786-39c622dd5cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:00.866032 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a1007-4ca4-48af-9786-39c622dd5cd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:00.866777 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a1007-4ca4-48af-9786-39c622dd5cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:00.868307 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a1007-4ca4-48af-9786-39c622dd5cd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:00.870609 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665a1007-4ca4-48af-9786-39c622dd5cd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:00.871868 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cecb1a-df07-4c7a-9c06-78571de9f37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:00.872331 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665a1007-4ca4-48af-9786-39c622dd5cd1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:00.873095 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cecb1a-df07-4c7a-9c06-78571de9f37e admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:00.873551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cecb1a-df07-4c7a-9c06-78571de9f37e admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:00.876830 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cecb1a-df07-4c7a-9c06-78571de9f37e admin admin] PATH_INFO: `/v3/projects/28c40fe75d6b455587f77add5ac6b474/users/d3076492ca0e4619acb39fcf9dc39b35/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:00.895434 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 814/1627] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:00.896595 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a1007-4ca4-48af-9786-39c622dd5cd1 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:00.921384 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a1007-4ca4-48af-9786-39c622dd5cd1 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:00.929220 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e42a22-c3dc-4a40-8065-170fb014ee10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:00.929936 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e42a22-c3dc-4a40-8065-170fb014ee10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:00.930222 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e42a22-c3dc-4a40-8065-170fb014ee10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:00.930504 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e42a22-c3dc-4a40-8065-170fb014ee10 admin admin] PATH_INFO: `/v3/projects/314d8d99f97549b0ab7f54664473b053/users/4f68963381b04ffea13a17b31af692a1/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:00.935111 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e42a22-c3dc-4a40-8065-170fb014ee10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=314d8d99f97549b0ab7f54664473b053, user_id=4f68963381b04ffea13a17b31af692a1, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:00.940936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cecb1a-df07-4c7a-9c06-78571de9f37e admin admin] RBAC: Authorizing `identity:create_grant(project_id=28c40fe75d6b455587f77add5ac6b474, user_id=d3076492ca0e4619acb39fcf9dc39b35, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:00.942044 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e42a22-c3dc-4a40-8065-170fb014ee10 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:00.942183 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cecb1a-df07-4c7a-9c06-78571de9f37e admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:00.965981 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 815/1628] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:00 2022] PUT /identity/v3/projects/314d8d99f97549b0ab7f54664473b053/users/4f68963381b04ffea13a17b31af692a1/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:28:00.976515 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff1de6-89b4-4c95-baee-e3e9b6765925 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:00.976862 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff1de6-89b4-4c95-baee-e3e9b6765925 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:00.977124 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff1de6-89b4-4c95-baee-e3e9b6765925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:01.003703 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 814/1629] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:00 2022] PUT /identity/v3/projects/28c40fe75d6b455587f77add5ac6b474/users/d3076492ca0e4619acb39fcf9dc39b35/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:01.004686 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cecb1a-df07-4c7a-9c06-78571de9f37e admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:01.036916 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-1bff1de6-89b4-4c95-baee-e3e9b6765925 None None] MFA Rules not processed for user `4f68963381b04ffea13a17b31af692a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:01.038137 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1bff1de6-89b4-4c95-baee-e3e9b6765925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:01.178578 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cecb1a-df07-4c7a-9c06-78571de9f37e admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:01.192145 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d448b-9c50-4489-aa08-5f3bfe97b8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:01.193768 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 816/1630] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:00 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:01.200555 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d448b-9c50-4489-aa08-5f3bfe97b8d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:01.202054 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d448b-9c50-4489-aa08-5f3bfe97b8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:01.202346 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d448b-9c50-4489-aa08-5f3bfe97b8d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:01.204011 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339d448b-9c50-4489-aa08-5f3bfe97b8d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:01.206237 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff1de6-89b4-4c95-baee-e3e9b6765925 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:01.209865 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339d448b-9c50-4489-aa08-5f3bfe97b8d4 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:01.261667 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 815/1631] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:01.262676 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d448b-9c50-4489-aa08-5f3bfe97b8d4 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:01.291684 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d448b-9c50-4489-aa08-5f3bfe97b8d4 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:01.318043 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3083ff5c-c169-4356-b9b6-00f00c941c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:01.321438 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3083ff5c-c169-4356-b9b6-00f00c941c99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:01.322340 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3083ff5c-c169-4356-b9b6-00f00c941c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:01.323180 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3083ff5c-c169-4356-b9b6-00f00c941c99 admin admin] PATH_INFO: `/v3/projects/28c40fe75d6b455587f77add5ac6b474/users/d3076492ca0e4619acb39fcf9dc39b35/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:01.332851 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3083ff5c-c169-4356-b9b6-00f00c941c99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28c40fe75d6b455587f77add5ac6b474, user_id=d3076492ca0e4619acb39fcf9dc39b35, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:01.338581 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3083ff5c-c169-4356-b9b6-00f00c941c99 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:01.350962 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff1de6-89b4-4c95-baee-e3e9b6765925 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:01.365010 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 816/1632] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:01 2022] PUT /identity/v3/projects/28c40fe75d6b455587f77add5ac6b474/users/d3076492ca0e4619acb39fcf9dc39b35/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:01.367109 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52170de7-672c-4fd3-a3d1-7d1ad4232ab8 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:01.375097 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52170de7-672c-4fd3-a3d1-7d1ad4232ab8 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:01.375545 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52170de7-672c-4fd3-a3d1-7d1ad4232ab8 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:01.390031 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 817/1633] 10.222.0.26 () {54 vars in 912 bytes} [Fri Sep 9 20:28:01 2022] GET /identity => generated 270 bytes in 24 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:28:01.394539 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43e29b-8420-4670-bd11-c93a1ae43b83 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:01.394864 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43e29b-8420-4670-bd11-c93a1ae43b83 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:01.395217 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43e29b-8420-4670-bd11-c93a1ae43b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:01.457187 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:01.460065 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a254b-22c3-428b-ba95-d87a19a881f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:01.463739 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a254b-22c3-428b-ba95-d87a19a881f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:01.463739 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a254b-22c3-428b-ba95-d87a19a881f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:01.463739 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a254b-22c3-428b-ba95-d87a19a881f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:01.521854 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-cf43e29b-8420-4670-bd11-c93a1ae43b83 None None] MFA Rules not processed for user `d3076492ca0e4619acb39fcf9dc39b35`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:01.525389 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-cf43e29b-8420-4670-bd11-c93a1ae43b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:01.556276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:01.685488 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 818/1634] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:01 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:01.687033 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e606ed-dfb3-4fda-80ed-d7e60606020d None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:01.688659 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e606ed-dfb3-4fda-80ed-d7e60606020d None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:01.689213 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e606ed-dfb3-4fda-80ed-d7e60606020d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:01.729785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-05e606ed-dfb3-4fda-80ed-d7e60606020d None None] MFA Rules not processed for user `87ece6128f9245f7b49f3e7a9faaed22`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:01.732646 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-05e606ed-dfb3-4fda-80ed-d7e60606020d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:01.779023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a254b-22c3-428b-ba95-d87a19a881f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:01.780157 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a254b-22c3-428b-ba95-d87a19a881f5 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:01.813384 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 817/1635] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:01 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 20:28:01.814038 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a254b-22c3-428b-ba95-d87a19a881f5 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:01.831540 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a254b-22c3-428b-ba95-d87a19a881f5 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:01.836173 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862c51f-301d-4bc1-8c84-0668ee78acdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:01.837039 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862c51f-301d-4bc1-8c84-0668ee78acdf service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:01.837552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862c51f-301d-4bc1-8c84-0668ee78acdf service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:01.837997 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862c51f-301d-4bc1-8c84-0668ee78acdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:01.850232 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7862c51f-301d-4bc1-8c84-0668ee78acdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:01.852292 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7862c51f-301d-4bc1-8c84-0668ee78acdf service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:01.870220 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 818/1636] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:01 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:01.882962 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862c51f-301d-4bc1-8c84-0668ee78acdf service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:01.916028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 819/1637] 10.222.0.26 () {58 vars in 1009 bytes} [Fri Sep 9 20:28:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:01.943610 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e606ed-dfb3-4fda-80ed-d7e60606020d None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:01.957112 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e606ed-dfb3-4fda-80ed-d7e60606020d None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:01.965918 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823a93c-66f8-4bbb-ba63-1b37b91d6add service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:01.967348 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823a93c-66f8-4bbb-ba63-1b37b91d6add service cinder] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:01.967637 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823a93c-66f8-4bbb-ba63-1b37b91d6add service cinder] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:01.969009 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823a93c-66f8-4bbb-ba63-1b37b91d6add service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:02.002923 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e823a93c-66f8-4bbb-ba63-1b37b91d6add service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:02.006585 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e823a93c-66f8-4bbb-ba63-1b37b91d6add service cinder] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:02.032209 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 820/1638] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 20:28:02.033313 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862c51f-301d-4bc1-8c84-0668ee78acdf service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:02.038297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9b6044-d972-4e5a-8eef-18f0d052cf4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:02.039030 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b6044-d972-4e5a-8eef-18f0d052cf4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:02.039314 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b6044-d972-4e5a-8eef-18f0d052cf4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:02.039680 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b6044-d972-4e5a-8eef-18f0d052cf4d admin admin] PATH_INFO: `/v3/users/5cfb93f9864f435ba544dbb08be07b1a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:02.040889 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9b6044-d972-4e5a-8eef-18f0d052cf4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cfb93f9864f435ba544dbb08be07b1a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:02.042020 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: WARNING py.warnings [None req-da9b6044-d972-4e5a-8eef-18f0d052cf4d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 20:28:02.042020 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: warnings.warn(msg) Sep 09 20:28:02.042020 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: Sep 09 20:28:02.043552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9b6044-d972-4e5a-8eef-18f0d052cf4d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:02.133703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-da9b6044-d972-4e5a-8eef-18f0d052cf4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cfb93f9864f435ba544dbb08be07b1a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:02.161557 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-da9b6044-d972-4e5a-8eef-18f0d052cf4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cfb93f9864f435ba544dbb08be07b1a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:02.211040 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-da9b6044-d972-4e5a-8eef-18f0d052cf4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cfb93f9864f435ba544dbb08be07b1a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:02.245789 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-da9b6044-d972-4e5a-8eef-18f0d052cf4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cfb93f9864f435ba544dbb08be07b1a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:02.246460 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-da9b6044-d972-4e5a-8eef-18f0d052cf4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cfb93f9864f435ba544dbb08be07b1a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:02.258671 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44025c21-8809-4639-816f-5fc262d4a82b None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:02.259190 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44025c21-8809-4639-816f-5fc262d4a82b None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:02.259491 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44025c21-8809-4639-816f-5fc262d4a82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:02.262393 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 819/1639] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:28:01 2022] DELETE /identity/v3/users/5cfb93f9864f435ba544dbb08be07b1a => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:02.292777 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-44025c21-8809-4639-816f-5fc262d4a82b None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:02.294814 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-44025c21-8809-4639-816f-5fc262d4a82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:02.302560 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9b6044-d972-4e5a-8eef-18f0d052cf4d admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:02.304316 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-da9b6044-d972-4e5a-8eef-18f0d052cf4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:02.382808 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:02.483078 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 821/1640] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:02.492314 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9b6044-d972-4e5a-8eef-18f0d052cf4d admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:02.496749 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44025c21-8809-4639-816f-5fc262d4a82b None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:02.498350 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-44025c21-8809-4639-816f-5fc262d4a82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:02.513183 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd49f177-8852-44d8-a28f-9c3e5e1b5d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:02.516147 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49f177-8852-44d8-a28f-9c3e5e1b5d8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:02.516414 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49f177-8852-44d8-a28f-9c3e5e1b5d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:02.516708 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49f177-8852-44d8-a28f-9c3e5e1b5d8e admin admin] PATH_INFO: `/v3/users/8b4fc11c58544a34b31de46351b5e91e` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:02.521338 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:02.524531 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd49f177-8852-44d8-a28f-9c3e5e1b5d8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b4fc11c58544a34b31de46351b5e91e)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:02.526662 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd49f177-8852-44d8-a28f-9c3e5e1b5d8e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:02.608638 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-cd49f177-8852-44d8-a28f-9c3e5e1b5d8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b4fc11c58544a34b31de46351b5e91e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:02.629841 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-cd49f177-8852-44d8-a28f-9c3e5e1b5d8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b4fc11c58544a34b31de46351b5e91e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:02.673417 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-cd49f177-8852-44d8-a28f-9c3e5e1b5d8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b4fc11c58544a34b31de46351b5e91e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:02.688294 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-cd49f177-8852-44d8-a28f-9c3e5e1b5d8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b4fc11c58544a34b31de46351b5e91e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:02.688960 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-cd49f177-8852-44d8-a28f-9c3e5e1b5d8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b4fc11c58544a34b31de46351b5e91e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:02.700552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 820/1641] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:28:02 2022] DELETE /identity/v3/users/8b4fc11c58544a34b31de46351b5e91e => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:02.702480 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb9510-a906-4af6-9107-08ac86f8c26f None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:02.703567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb9510-a906-4af6-9107-08ac86f8c26f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:02.703906 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb9510-a906-4af6-9107-08ac86f8c26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:02.750377 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-d9bb9510-a906-4af6-9107-08ac86f8c26f None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:02.751214 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44025c21-8809-4639-816f-5fc262d4a82b None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:02.754230 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d9bb9510-a906-4af6-9107-08ac86f8c26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:02.891774 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411545c-fcad-4efc-b535-ac0f7b42d2a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:02.892337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411545c-fcad-4efc-b535-ac0f7b42d2a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:02.892616 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411545c-fcad-4efc-b535-ac0f7b42d2a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:02.892887 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411545c-fcad-4efc-b535-ac0f7b42d2a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:02.896083 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-4411545c-fcad-4efc-b535-ac0f7b42d2a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:02.946286 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 821/1642] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:02.972933 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b8863-da10-4173-908a-380ca15f2e3b None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:02.974128 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b8863-da10-4173-908a-380ca15f2e3b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:02.974690 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b8863-da10-4173-908a-380ca15f2e3b None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:02.978331 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 822/1643] 10.222.0.26 () {54 vars in 962 bytes} [Fri Sep 9 20:28:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:28:02.987556 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b8863-da10-4173-908a-380ca15f2e3b None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:02.993657 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-339b8863-da10-4173-908a-380ca15f2e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:03.044458 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b8863-da10-4173-908a-380ca15f2e3b None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:03.057160 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c80560-58c7-4972-ac33-4c2977ad9b8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:03.058270 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c80560-58c7-4972-ac33-4c2977ad9b8d service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:03.058865 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c80560-58c7-4972-ac33-4c2977ad9b8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:03.059442 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c80560-58c7-4972-ac33-4c2977ad9b8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:03.074245 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411545c-fcad-4efc-b535-ac0f7b42d2a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:03.076050 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411545c-fcad-4efc-b535-ac0f7b42d2a7 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:03.080736 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c80560-58c7-4972-ac33-4c2977ad9b8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:03.083425 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c80560-58c7-4972-ac33-4c2977ad9b8d service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:03.101636 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 823/1644] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:03.102891 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 822/1645] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:03.105966 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d735a14-75bd-42eb-bcd9-4caa8e465e87 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:03.107206 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d735a14-75bd-42eb-bcd9-4caa8e465e87 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:03.110516 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d735a14-75bd-42eb-bcd9-4caa8e465e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:03.173367 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-8d735a14-75bd-42eb-bcd9-4caa8e465e87 None None] MFA Rules not processed for user `87ece6128f9245f7b49f3e7a9faaed22`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:03.175199 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8d735a14-75bd-42eb-bcd9-4caa8e465e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:03.228013 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:03.327461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 824/1646] 10.222.0.26 () {58 vars in 1059 bytes} [Fri Sep 9 20:28:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:03.333015 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57873dc1-1758-4460-9adc-b5df25b45079 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:03.354137 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57873dc1-1758-4460-9adc-b5df25b45079 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:03.354137 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57873dc1-1758-4460-9adc-b5df25b45079 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:03.354137 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 823/1647] 10.222.0.26 () {54 vars in 912 bytes} [Fri Sep 9 20:28:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:28:03.367447 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed49b3-9173-4a5c-8001-36e5e35c3251 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:03.367747 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed49b3-9173-4a5c-8001-36e5e35c3251 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:03.368016 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed49b3-9173-4a5c-8001-36e5e35c3251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:03.406792 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-15ed49b3-9173-4a5c-8001-36e5e35c3251 None None] MFA Rules not processed for user `87ece6128f9245f7b49f3e7a9faaed22`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:03.408039 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-15ed49b3-9173-4a5c-8001-36e5e35c3251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:03.438989 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 825/1648] 10.222.0.26 () {58 vars in 1009 bytes} [Fri Sep 9 20:28:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:28:03.460695 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57873dc1-1758-4460-9adc-b5df25b45079 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:03.462517 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-57873dc1-1758-4460-9adc-b5df25b45079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:03.596107 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57873dc1-1758-4460-9adc-b5df25b45079 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:03.604783 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aadeb28-7ca8-4874-a03c-688a8c4ad89b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:03.605462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aadeb28-7ca8-4874-a03c-688a8c4ad89b service cinder] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:03.609088 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aadeb28-7ca8-4874-a03c-688a8c4ad89b service cinder] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:03.609449 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aadeb28-7ca8-4874-a03c-688a8c4ad89b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:03.634872 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aadeb28-7ca8-4874-a03c-688a8c4ad89b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:03.639859 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aadeb28-7ca8-4874-a03c-688a8c4ad89b service cinder] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:03.685077 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 824/1649] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:04.882802 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed49b3-9173-4a5c-8001-36e5e35c3251 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:04.884882 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-15ed49b3-9173-4a5c-8001-36e5e35c3251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:04.920469 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed49b3-9173-4a5c-8001-36e5e35c3251 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:04.935742 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a4915-c41e-4b3d-9a59-3c55d020355a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:04.936514 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a4915-c41e-4b3d-9a59-3c55d020355a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:04.937841 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a4915-c41e-4b3d-9a59-3c55d020355a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:04.940134 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a4915-c41e-4b3d-9a59-3c55d020355a admin admin] PATH_INFO: `/v3/projects/a559fc57f6444f3088f93ed92fb8afc7` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:04.942159 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a4915-c41e-4b3d-9a59-3c55d020355a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a559fc57f6444f3088f93ed92fb8afc7)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:04.948168 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a4915-c41e-4b3d-9a59-3c55d020355a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:05.078297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9a2a4915-c41e-4b3d-9a59-3c55d020355a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a559fc57f6444f3088f93ed92fb8afc7', 'request_id': 'req-9a2a4915-c41e-4b3d-9a59-3c55d020355a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:05.093220 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9a2a4915-c41e-4b3d-9a59-3c55d020355a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a559fc57f6444f3088f93ed92fb8afc7', 'request_id': 'req-9a2a4915-c41e-4b3d-9a59-3c55d020355a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:05.106417 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9a2a4915-c41e-4b3d-9a59-3c55d020355a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a559fc57f6444f3088f93ed92fb8afc7', 'request_id': 'req-9a2a4915-c41e-4b3d-9a59-3c55d020355a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:05.125977 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9a2a4915-c41e-4b3d-9a59-3c55d020355a admin admin] The token cache is being invalidate because project a559fc57f6444f3088f93ed92fb8afc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:28:05.126646 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9a2a4915-c41e-4b3d-9a59-3c55d020355a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:05.131041 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9a2a4915-c41e-4b3d-9a59-3c55d020355a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:05.136885 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 826/1650] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:28:04 2022] DELETE /identity/v3/projects/a559fc57f6444f3088f93ed92fb8afc7 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:05.140007 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aadeb28-7ca8-4874-a03c-688a8c4ad89b service cinder] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:05.154111 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aadeb28-7ca8-4874-a03c-688a8c4ad89b service cinder] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:05.173850 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:05.265299 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a4915-c41e-4b3d-9a59-3c55d020355a admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:05.267037 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9a2a4915-c41e-4b3d-9a59-3c55d020355a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:05.283838 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e77d29-68a2-4775-a880-0f7d415552f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:05.288589 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e77d29-68a2-4775-a880-0f7d415552f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:05.288589 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e77d29-68a2-4775-a880-0f7d415552f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:05.288784 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e77d29-68a2-4775-a880-0f7d415552f1 admin admin] PATH_INFO: `/v3/users/4f68963381b04ffea13a17b31af692a1` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:05.295086 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e77d29-68a2-4775-a880-0f7d415552f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f68963381b04ffea13a17b31af692a1)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:05.298553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a4915-c41e-4b3d-9a59-3c55d020355a admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:05.302094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e77d29-68a2-4775-a880-0f7d415552f1 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:05.311999 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cfd0eb-50d2-4616-8487-9f1e369d525c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:05.313085 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cfd0eb-50d2-4616-8487-9f1e369d525c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:05.313390 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cfd0eb-50d2-4616-8487-9f1e369d525c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:05.315581 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cfd0eb-50d2-4616-8487-9f1e369d525c admin admin] PATH_INFO: `/v3/projects/a8041d5ff2ad45fc8e7577b624585332` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:05.319170 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cfd0eb-50d2-4616-8487-9f1e369d525c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8041d5ff2ad45fc8e7577b624585332)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:05.324881 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cfd0eb-50d2-4616-8487-9f1e369d525c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:05.415924 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-07cfd0eb-50d2-4616-8487-9f1e369d525c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8041d5ff2ad45fc8e7577b624585332', 'request_id': 'req-07cfd0eb-50d2-4616-8487-9f1e369d525c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:05.423672 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-07cfd0eb-50d2-4616-8487-9f1e369d525c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8041d5ff2ad45fc8e7577b624585332', 'request_id': 'req-07cfd0eb-50d2-4616-8487-9f1e369d525c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:05.431522 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-08e77d29-68a2-4775-a880-0f7d415552f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f68963381b04ffea13a17b31af692a1'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:05.447815 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-07cfd0eb-50d2-4616-8487-9f1e369d525c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8041d5ff2ad45fc8e7577b624585332', 'request_id': 'req-07cfd0eb-50d2-4616-8487-9f1e369d525c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:05.453619 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-08e77d29-68a2-4775-a880-0f7d415552f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f68963381b04ffea13a17b31af692a1'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:05.469643 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-08e77d29-68a2-4775-a880-0f7d415552f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f68963381b04ffea13a17b31af692a1'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:05.474804 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-07cfd0eb-50d2-4616-8487-9f1e369d525c admin admin] The token cache is being invalidate because project a8041d5ff2ad45fc8e7577b624585332 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:28:05.475818 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-07cfd0eb-50d2-4616-8487-9f1e369d525c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:05.476859 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-07cfd0eb-50d2-4616-8487-9f1e369d525c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:05.480874 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 827/1651] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:28:05 2022] DELETE /identity/v3/projects/a8041d5ff2ad45fc8e7577b624585332 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:05.486547 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-08e77d29-68a2-4775-a880-0f7d415552f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f68963381b04ffea13a17b31af692a1'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:05.487562 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cfd0eb-50d2-4616-8487-9f1e369d525c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:05.487824 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-08e77d29-68a2-4775-a880-0f7d415552f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f68963381b04ffea13a17b31af692a1'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:05.490662 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-07cfd0eb-50d2-4616-8487-9f1e369d525c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:05.500622 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 825/1652] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:28:05 2022] DELETE /identity/v3/users/4f68963381b04ffea13a17b31af692a1 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:05.543963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:05.633403 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cfd0eb-50d2-4616-8487-9f1e369d525c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:05.727724 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b786b0-c90d-4cdd-9014-feadf9d05caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:05.729102 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b786b0-c90d-4cdd-9014-feadf9d05caa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:05.729552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b786b0-c90d-4cdd-9014-feadf9d05caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:05.730122 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b786b0-c90d-4cdd-9014-feadf9d05caa admin admin] PATH_INFO: `/v3/users/d3076492ca0e4619acb39fcf9dc39b35` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:05.731567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b786b0-c90d-4cdd-9014-feadf9d05caa admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3076492ca0e4619acb39fcf9dc39b35)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:05.735009 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b786b0-c90d-4cdd-9014-feadf9d05caa admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:05.848882 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-34b786b0-c90d-4cdd-9014-feadf9d05caa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3076492ca0e4619acb39fcf9dc39b35'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:05.867225 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-34b786b0-c90d-4cdd-9014-feadf9d05caa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3076492ca0e4619acb39fcf9dc39b35'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:05.883359 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-34b786b0-c90d-4cdd-9014-feadf9d05caa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3076492ca0e4619acb39fcf9dc39b35'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:05.918670 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-34b786b0-c90d-4cdd-9014-feadf9d05caa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3076492ca0e4619acb39fcf9dc39b35'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:05.919812 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-34b786b0-c90d-4cdd-9014-feadf9d05caa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3076492ca0e4619acb39fcf9dc39b35'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:05.929637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 828/1653] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:28:05 2022] DELETE /identity/v3/users/d3076492ca0e4619acb39fcf9dc39b35 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:07.742601 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e77d29-68a2-4775-a880-0f7d415552f1 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:07.745449 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-08e77d29-68a2-4775-a880-0f7d415552f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:07.779581 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e77d29-68a2-4775-a880-0f7d415552f1 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:08.028429 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c26306-572b-4300-a2de-9ae62250198b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:08.029729 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c26306-572b-4300-a2de-9ae62250198b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:08.029962 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c26306-572b-4300-a2de-9ae62250198b admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:08.030237 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c26306-572b-4300-a2de-9ae62250198b admin admin] PATH_INFO: `/v3/projects/314d8d99f97549b0ab7f54664473b053` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:08.031486 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c26306-572b-4300-a2de-9ae62250198b admin admin] RBAC: Authorizing `identity:delete_project(project_id=314d8d99f97549b0ab7f54664473b053)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:08.032622 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: WARNING py.warnings [None req-e3c26306-572b-4300-a2de-9ae62250198b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 20:28:08.032622 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: warnings.warn(msg) Sep 09 20:28:08.032622 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: Sep 09 20:28:08.033977 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c26306-572b-4300-a2de-9ae62250198b admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:08.047788 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b786b0-c90d-4cdd-9014-feadf9d05caa admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:08.050386 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-34b786b0-c90d-4cdd-9014-feadf9d05caa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:08.075637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b786b0-c90d-4cdd-9014-feadf9d05caa admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:08.086910 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d391de42-a8a3-4b24-a778-8a96b4bee9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:08.087585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391de42-a8a3-4b24-a778-8a96b4bee9be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:08.088020 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391de42-a8a3-4b24-a778-8a96b4bee9be admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:08.088369 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391de42-a8a3-4b24-a778-8a96b4bee9be admin admin] PATH_INFO: `/v3/projects/28c40fe75d6b455587f77add5ac6b474` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:08.089744 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d391de42-a8a3-4b24-a778-8a96b4bee9be admin admin] RBAC: Authorizing `identity:delete_project(project_id=28c40fe75d6b455587f77add5ac6b474)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:08.107263 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d391de42-a8a3-4b24-a778-8a96b4bee9be admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:08.146257 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:08.183442 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-e3c26306-572b-4300-a2de-9ae62250198b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '314d8d99f97549b0ab7f54664473b053', 'request_id': 'req-e3c26306-572b-4300-a2de-9ae62250198b'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:08.209989 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-e3c26306-572b-4300-a2de-9ae62250198b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '314d8d99f97549b0ab7f54664473b053', 'request_id': 'req-e3c26306-572b-4300-a2de-9ae62250198b'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:08.223047 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-e3c26306-572b-4300-a2de-9ae62250198b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '314d8d99f97549b0ab7f54664473b053', 'request_id': 'req-e3c26306-572b-4300-a2de-9ae62250198b'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:08.224594 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:08.228499 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d391de42-a8a3-4b24-a778-8a96b4bee9be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28c40fe75d6b455587f77add5ac6b474', 'request_id': 'req-d391de42-a8a3-4b24-a778-8a96b4bee9be'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:08.242572 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d391de42-a8a3-4b24-a778-8a96b4bee9be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28c40fe75d6b455587f77add5ac6b474', 'request_id': 'req-d391de42-a8a3-4b24-a778-8a96b4bee9be'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:08.252475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d391de42-a8a3-4b24-a778-8a96b4bee9be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28c40fe75d6b455587f77add5ac6b474', 'request_id': 'req-d391de42-a8a3-4b24-a778-8a96b4bee9be'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:08.269585 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-e3c26306-572b-4300-a2de-9ae62250198b admin admin] The token cache is being invalidate because project 314d8d99f97549b0ab7f54664473b053 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:28:08.269900 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-e3c26306-572b-4300-a2de-9ae62250198b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:08.271453 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-e3c26306-572b-4300-a2de-9ae62250198b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:08.274369 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 826/1654] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:28:07 2022] DELETE /identity/v3/projects/314d8d99f97549b0ab7f54664473b053 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:08.283042 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e57a7-a922-4d33-93b9-2d38f9e749c8 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:08.283347 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e57a7-a922-4d33-93b9-2d38f9e749c8 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:08.283623 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e57a7-a922-4d33-93b9-2d38f9e749c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:08.285959 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d391de42-a8a3-4b24-a778-8a96b4bee9be admin admin] The token cache is being invalidate because project 28c40fe75d6b455587f77add5ac6b474 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:28:08.286345 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d391de42-a8a3-4b24-a778-8a96b4bee9be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:08.287252 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d391de42-a8a3-4b24-a778-8a96b4bee9be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:08.289812 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 829/1655] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:28:08 2022] DELETE /identity/v3/projects/28c40fe75d6b455587f77add5ac6b474 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:08.294963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565173af-42f4-4955-8a6b-807a56390ef2 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:08.295428 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565173af-42f4-4955-8a6b-807a56390ef2 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:08.303036 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565173af-42f4-4955-8a6b-807a56390ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:08.332545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-565173af-42f4-4955-8a6b-807a56390ef2 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:08.334778 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-565173af-42f4-4955-8a6b-807a56390ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:08.351513 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-a64e57a7-a922-4d33-93b9-2d38f9e749c8 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:08.352855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a64e57a7-a922-4d33-93b9-2d38f9e749c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:08.533092 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 827/1656] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:08.542240 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a856d754-6a7d-4fbe-b79b-230020f7e1e6 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:08.542544 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a856d754-6a7d-4fbe-b79b-230020f7e1e6 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:08.542792 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a856d754-6a7d-4fbe-b79b-230020f7e1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:08.562875 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 830/1657] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:28:08.564429 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e6cd62-0d41-487f-a885-e523e1fa8632 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:08.564718 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e6cd62-0d41-487f-a885-e523e1fa8632 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:08.565004 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e6cd62-0d41-487f-a885-e523e1fa8632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:08.633817 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-a856d754-6a7d-4fbe-b79b-230020f7e1e6 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:08.642060 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a856d754-6a7d-4fbe-b79b-230020f7e1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:08.669028 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-41e6cd62-0d41-487f-a885-e523e1fa8632 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:08.670368 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-41e6cd62-0d41-487f-a885-e523e1fa8632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:08.678742 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 828/1658] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:28:08.682775 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a856d754-6a7d-4fbe-b79b-230020f7e1e6 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:08.716299 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 831/1659] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:08.717156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e6cd62-0d41-487f-a885-e523e1fa8632 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:08.718794 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a856d754-6a7d-4fbe-b79b-230020f7e1e6 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:08.747427 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e6cd62-0d41-487f-a885-e523e1fa8632 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:08.749209 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61bc8f-775c-41a6-8098-395fa78866ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:08.750092 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61bc8f-775c-41a6-8098-395fa78866ae admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:08.753674 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61bc8f-775c-41a6-8098-395fa78866ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:08.754661 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61bc8f-775c-41a6-8098-395fa78866ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:08.759692 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec61bc8f-775c-41a6-8098-395fa78866ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:28:08.759991 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec61bc8f-775c-41a6-8098-395fa78866ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:08.764131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec61bc8f-775c-41a6-8098-395fa78866ae admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:08.784061 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c63bd-1bb8-4a11-aa69-8b5cfd42a8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:08.785127 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c63bd-1bb8-4a11-aa69-8b5cfd42a8a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:08.785620 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c63bd-1bb8-4a11-aa69-8b5cfd42a8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:08.786095 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c63bd-1bb8-4a11-aa69-8b5cfd42a8a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:08.789389 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c63bd-1bb8-4a11-aa69-8b5cfd42a8a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:28:08.789922 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c63bd-1bb8-4a11-aa69-8b5cfd42a8a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:08.792993 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c63bd-1bb8-4a11-aa69-8b5cfd42a8a0 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:08.819322 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 829/1660] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:28:08.820294 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61bc8f-775c-41a6-8098-395fa78866ae admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:08.840225 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 832/1661] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:28:08.844790 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c63bd-1bb8-4a11-aa69-8b5cfd42a8a0 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:08.859202 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61bc8f-775c-41a6-8098-395fa78866ae admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:08.861400 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c63bd-1bb8-4a11-aa69-8b5cfd42a8a0 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:08.870998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f1565-5391-4fa3-be69-691c2dd147b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:08.872367 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f1565-5391-4fa3-be69-691c2dd147b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:08.873024 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f1565-5391-4fa3-be69-691c2dd147b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:08.873525 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f1565-5391-4fa3-be69-691c2dd147b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:08.875029 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f1565-5391-4fa3-be69-691c2dd147b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:08.878010 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f1565-5391-4fa3-be69-691c2dd147b6 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:08.878247 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16aa7b5-f223-4f71-8482-c317fa25a238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:08.878843 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16aa7b5-f223-4f71-8482-c317fa25a238 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:08.879128 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16aa7b5-f223-4f71-8482-c317fa25a238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:08.879392 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16aa7b5-f223-4f71-8482-c317fa25a238 admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:08.882510 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16aa7b5-f223-4f71-8482-c317fa25a238 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:08.884649 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16aa7b5-f223-4f71-8482-c317fa25a238 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:08.954491 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 833/1662] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:28:08 2022] POST /identity/v3/projects => generated 381 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:08.958029 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f1565-5391-4fa3-be69-691c2dd147b6 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:08.966509 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 830/1663] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:28:08 2022] POST /identity/v3/projects => generated 387 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:08.969054 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16aa7b5-f223-4f71-8482-c317fa25a238 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:09.151644 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16aa7b5-f223-4f71-8482-c317fa25a238 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:09.170390 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c5598c-8dad-4cde-8227-11027b6019ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:09.171045 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c5598c-8dad-4cde-8227-11027b6019ba admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:09.171364 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c5598c-8dad-4cde-8227-11027b6019ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:09.171718 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c5598c-8dad-4cde-8227-11027b6019ba admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:09.172817 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c5598c-8dad-4cde-8227-11027b6019ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:09.175118 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c5598c-8dad-4cde-8227-11027b6019ba admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:09.221121 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 831/1664] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:28:08 2022] POST /identity/v3/users => generated 294 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:09.225442 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c5598c-8dad-4cde-8227-11027b6019ba admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:09.238101 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c5598c-8dad-4cde-8227-11027b6019ba admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:09.244117 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53297b5a-1063-4cb1-ad9f-39fe6054888c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:09.244922 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53297b5a-1063-4cb1-ad9f-39fe6054888c admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:09.245867 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53297b5a-1063-4cb1-ad9f-39fe6054888c admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:09.247177 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53297b5a-1063-4cb1-ad9f-39fe6054888c admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:09.247632 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53297b5a-1063-4cb1-ad9f-39fe6054888c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:09.250606 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53297b5a-1063-4cb1-ad9f-39fe6054888c admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:09.256653 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f1565-5391-4fa3-be69-691c2dd147b6 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:09.271468 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e16e88-8a4e-451f-9acf-c8b6ad582bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:09.272278 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e16e88-8a4e-451f-9acf-c8b6ad582bec admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:09.272661 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e16e88-8a4e-451f-9acf-c8b6ad582bec admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:09.273036 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e16e88-8a4e-451f-9acf-c8b6ad582bec admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:09.273805 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e16e88-8a4e-451f-9acf-c8b6ad582bec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:09.277069 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e16e88-8a4e-451f-9acf-c8b6ad582bec admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:09.282224 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 832/1665] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:09.284007 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53297b5a-1063-4cb1-ad9f-39fe6054888c admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:09.301544 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:09.322305 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53297b5a-1063-4cb1-ad9f-39fe6054888c admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:09.324340 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 834/1666] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:28:08 2022] POST /identity/v3/users => generated 291 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:09.325478 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e16e88-8a4e-451f-9acf-c8b6ad582bec admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:09.344664 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2004c4a3-79d4-4445-aaa2-d90cfc4182ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:09.345330 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004c4a3-79d4-4445-aaa2-d90cfc4182ca admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:09.345637 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004c4a3-79d4-4445-aaa2-d90cfc4182ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:09.345901 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004c4a3-79d4-4445-aaa2-d90cfc4182ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:09.348962 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e16e88-8a4e-451f-9acf-c8b6ad582bec admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:09.351476 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2004c4a3-79d4-4445-aaa2-d90cfc4182ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:09.355795 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2004c4a3-79d4-4445-aaa2-d90cfc4182ca admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:09.361608 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0e28c3-2f55-4fe0-ae68-4359b80fc90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:09.362440 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e28c3-2f55-4fe0-ae68-4359b80fc90a admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:09.363158 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e28c3-2f55-4fe0-ae68-4359b80fc90a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:09.363158 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e28c3-2f55-4fe0-ae68-4359b80fc90a admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:09.366010 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0e28c3-2f55-4fe0-ae68-4359b80fc90a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:09.367972 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0e28c3-2f55-4fe0-ae68-4359b80fc90a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:09.390414 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 835/1667] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:09.397887 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0e28c3-2f55-4fe0-ae68-4359b80fc90a admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:09.398076 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 833/1668] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:09.398674 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2004c4a3-79d4-4445-aaa2-d90cfc4182ca admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:09.429541 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0e28c3-2f55-4fe0-ae68-4359b80fc90a admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:09.438239 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11700811-ddcb-42e2-a6ca-53c71ebe4bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:09.439004 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11700811-ddcb-42e2-a6ca-53c71ebe4bfa admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:09.439384 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11700811-ddcb-42e2-a6ca-53c71ebe4bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:09.439759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11700811-ddcb-42e2-a6ca-53c71ebe4bfa admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:09.440509 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11700811-ddcb-42e2-a6ca-53c71ebe4bfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:09.442433 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11700811-ddcb-42e2-a6ca-53c71ebe4bfa admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:09.449168 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:09.459234 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 836/1669] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:09.460356 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11700811-ddcb-42e2-a6ca-53c71ebe4bfa admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:09.473879 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11700811-ddcb-42e2-a6ca-53c71ebe4bfa admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:09.474887 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2004c4a3-79d4-4445-aaa2-d90cfc4182ca admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:09.488682 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08ded2-a426-4cbf-b123-0d9388642fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:09.488682 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08ded2-a426-4cbf-b123-0d9388642fde admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:09.488682 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08ded2-a426-4cbf-b123-0d9388642fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:09.490509 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08ded2-a426-4cbf-b123-0d9388642fde admin admin] PATH_INFO: `/v3/projects/d15626d86387448eb1d19803feb3851d/users/0b6c61dc11e04ea8b3492287d52b615a/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:09.500850 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d66527-7ff8-46f4-8b6f-3c672d26f6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:09.502272 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d66527-7ff8-46f4-8b6f-3c672d26f6c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:09.502628 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d66527-7ff8-46f4-8b6f-3c672d26f6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:09.502975 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d66527-7ff8-46f4-8b6f-3c672d26f6c0 admin admin] PATH_INFO: `/v3/projects/4a7c343e5c21424099577ade88f2f903/users/23134965e73a4d4a89e8853f9d49ecab/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:09.521639 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd08ded2-a426-4cbf-b123-0d9388642fde admin admin] RBAC: Authorizing `identity:create_grant(project_id=d15626d86387448eb1d19803feb3851d, user_id=0b6c61dc11e04ea8b3492287d52b615a, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:09.524501 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd08ded2-a426-4cbf-b123-0d9388642fde admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:09.537768 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d66527-7ff8-46f4-8b6f-3c672d26f6c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a7c343e5c21424099577ade88f2f903, user_id=23134965e73a4d4a89e8853f9d49ecab, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:09.541695 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d66527-7ff8-46f4-8b6f-3c672d26f6c0 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:09.568841 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 834/1670] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:09 2022] PUT /identity/v3/projects/d15626d86387448eb1d19803feb3851d/users/0b6c61dc11e04ea8b3492287d52b615a/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:09.572080 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd62c6-79c6-4f13-860c-8878ae8d81b5 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:09.572551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd62c6-79c6-4f13-860c-8878ae8d81b5 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:09.572778 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 837/1671] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:09 2022] PUT /identity/v3/projects/4a7c343e5c21424099577ade88f2f903/users/23134965e73a4d4a89e8853f9d49ecab/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:09.573177 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd62c6-79c6-4f13-860c-8878ae8d81b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:09.594935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4a1ab-c66b-4c4d-8b55-056af2c333b3 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:09.598661 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4a1ab-c66b-4c4d-8b55-056af2c333b3 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:09.598981 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4a1ab-c66b-4c4d-8b55-056af2c333b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:09.625279 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-1bdd62c6-79c6-4f13-860c-8878ae8d81b5 None None] MFA Rules not processed for user `0b6c61dc11e04ea8b3492287d52b615a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:09.626776 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-1bdd62c6-79c6-4f13-860c-8878ae8d81b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:09.684461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-7cf4a1ab-c66b-4c4d-8b55-056af2c333b3 None None] MFA Rules not processed for user `23134965e73a4d4a89e8853f9d49ecab`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:09.686197 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7cf4a1ab-c66b-4c4d-8b55-056af2c333b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:09.811476 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 835/1672] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:09 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:28:09.813935 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd62c6-79c6-4f13-860c-8878ae8d81b5 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:09.894511 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 838/1673] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:09.898927 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf4a1ab-c66b-4c4d-8b55-056af2c333b3 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:10.025277 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd62c6-79c6-4f13-860c-8878ae8d81b5 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:10.045819 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d023f52-e9a9-4b2d-9da9-a216485072a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:10.046487 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d023f52-e9a9-4b2d-9da9-a216485072a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:10.046762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d023f52-e9a9-4b2d-9da9-a216485072a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:10.047031 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d023f52-e9a9-4b2d-9da9-a216485072a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:10.053030 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d023f52-e9a9-4b2d-9da9-a216485072a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:10.055877 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d023f52-e9a9-4b2d-9da9-a216485072a2 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:10.113108 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf4a1ab-c66b-4c4d-8b55-056af2c333b3 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:10.127826 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 836/1674] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:28:09 2022] POST /identity/v3/projects => generated 387 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:10.129559 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d023f52-e9a9-4b2d-9da9-a216485072a2 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:10.141862 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5170f-358f-4067-a1dd-7a0b72ad180c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:10.142501 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5170f-358f-4067-a1dd-7a0b72ad180c admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:10.142788 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5170f-358f-4067-a1dd-7a0b72ad180c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:10.143062 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5170f-358f-4067-a1dd-7a0b72ad180c admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:10.143714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db5170f-358f-4067-a1dd-7a0b72ad180c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:10.150002 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db5170f-358f-4067-a1dd-7a0b72ad180c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:10.226666 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:10.237558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 839/1675] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:28:09 2022] POST /identity/v3/projects => generated 379 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:28:10.241567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5170f-358f-4067-a1dd-7a0b72ad180c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:10.262810 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d023f52-e9a9-4b2d-9da9-a216485072a2 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:10.361325 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:10.386588 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5170f-358f-4067-a1dd-7a0b72ad180c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:10.403223 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e04ca-a631-4433-96e7-f436442e4972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:10.404376 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e04ca-a631-4433-96e7-f436442e4972 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:10.404376 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e04ca-a631-4433-96e7-f436442e4972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:10.404624 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e04ca-a631-4433-96e7-f436442e4972 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:10.405284 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09e04ca-a631-4433-96e7-f436442e4972 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:10.410171 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09e04ca-a631-4433-96e7-f436442e4972 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:10.438309 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b4933f-5bca-40a4-b0c7-d3a1795a266a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:10.441862 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4933f-5bca-40a4-b0c7-d3a1795a266a admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:10.442482 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4933f-5bca-40a4-b0c7-d3a1795a266a admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:10.443818 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4933f-5bca-40a4-b0c7-d3a1795a266a admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:10.447018 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b4933f-5bca-40a4-b0c7-d3a1795a266a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:10.458173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b4933f-5bca-40a4-b0c7-d3a1795a266a admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:10.484834 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 840/1676] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:28:10 2022] POST /identity/v3/users => generated 296 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:10.489322 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e04ca-a631-4433-96e7-f436442e4972 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:10.529663 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 837/1677] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:28:10 2022] POST /identity/v3/users => generated 300 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:10.533516 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac2bb4-4c16-4362-8be8-a48a471fe61d None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:10.537417 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac2bb4-4c16-4362-8be8-a48a471fe61d None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:10.537891 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac2bb4-4c16-4362-8be8-a48a471fe61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:10.555671 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e04ca-a631-4433-96e7-f436442e4972 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:10.562693 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a775fdca-a7c8-484c-9e8c-e905c25c7e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:10.563332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775fdca-a7c8-484c-9e8c-e905c25c7e8e admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:10.563612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775fdca-a7c8-484c-9e8c-e905c25c7e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:10.563888 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775fdca-a7c8-484c-9e8c-e905c25c7e8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:10.564539 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a775fdca-a7c8-484c-9e8c-e905c25c7e8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:10.566898 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a775fdca-a7c8-484c-9e8c-e905c25c7e8e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:10.589563 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 841/1678] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:10.590998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a775fdca-a7c8-484c-9e8c-e905c25c7e8e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:10.606573 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a775fdca-a7c8-484c-9e8c-e905c25c7e8e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:10.615497 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e0e04-d123-48d1-be41-5ee8d888b56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:10.616939 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e0e04-d123-48d1-be41-5ee8d888b56d admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:10.617399 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e0e04-d123-48d1-be41-5ee8d888b56d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:10.617746 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e0e04-d123-48d1-be41-5ee8d888b56d admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:10.619588 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5e0e04-d123-48d1-be41-5ee8d888b56d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:10.622202 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5e0e04-d123-48d1-be41-5ee8d888b56d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:10.627219 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-30ac2bb4-4c16-4362-8be8-a48a471fe61d None None] MFA Rules not processed for user `95bc48b6541c4388b45435036a1c4525`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:10.630002 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-30ac2bb4-4c16-4362-8be8-a48a471fe61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:10.643421 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 842/1679] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:10.650563 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e0e04-d123-48d1-be41-5ee8d888b56d admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:10.677205 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e0e04-d123-48d1-be41-5ee8d888b56d admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:10.689853 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909607f5-fb38-4572-808f-a0ac9cd1229b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:10.691298 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909607f5-fb38-4572-808f-a0ac9cd1229b admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:10.691758 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909607f5-fb38-4572-808f-a0ac9cd1229b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:10.692038 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909607f5-fb38-4572-808f-a0ac9cd1229b admin admin] PATH_INFO: `/v3/projects/52becdc909ff44c4bf026a2ff9ee165a/users/bf706de518bd47308323f07f47039b9f/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:10.750401 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909607f5-fb38-4572-808f-a0ac9cd1229b admin admin] RBAC: Authorizing `identity:create_grant(project_id=52becdc909ff44c4bf026a2ff9ee165a, user_id=bf706de518bd47308323f07f47039b9f, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:10.753298 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909607f5-fb38-4572-808f-a0ac9cd1229b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:10.831257 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 843/1680] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:10 2022] PUT /identity/v3/projects/52becdc909ff44c4bf026a2ff9ee165a/users/bf706de518bd47308323f07f47039b9f/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:10.832571 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909607f5-fb38-4572-808f-a0ac9cd1229b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:10.835115 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 838/1681] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:10.836638 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac2bb4-4c16-4362-8be8-a48a471fe61d None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:10.871501 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac2bb4-4c16-4362-8be8-a48a471fe61d None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:11.010932 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c407de38-abf1-4f52-aa63-0fb5922d3947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:11.011816 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407de38-abf1-4f52-aa63-0fb5922d3947 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:11.012207 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407de38-abf1-4f52-aa63-0fb5922d3947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:11.012604 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407de38-abf1-4f52-aa63-0fb5922d3947 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:11.013613 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c407de38-abf1-4f52-aa63-0fb5922d3947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:11.019229 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909607f5-fb38-4572-808f-a0ac9cd1229b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:11.019579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c407de38-abf1-4f52-aa63-0fb5922d3947 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:11.037401 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6283cf91-f87f-48de-a858-0466c88aa4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:11.039372 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283cf91-f87f-48de-a858-0466c88aa4eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:11.039805 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283cf91-f87f-48de-a858-0466c88aa4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:11.040343 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283cf91-f87f-48de-a858-0466c88aa4eb admin admin] PATH_INFO: `/v3/projects/7231b818f44b4cffa5274ed572974963/users/2211a2bf3fa045e48ed019135bd89888/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:11.043988 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 839/1682] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:11.043988 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c407de38-abf1-4f52-aa63-0fb5922d3947 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:11.051384 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-c407de38-abf1-4f52-aa63-0fb5922d3947 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:11.067641 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:11.109307 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6283cf91-f87f-48de-a858-0466c88aa4eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7231b818f44b4cffa5274ed572974963, user_id=2211a2bf3fa045e48ed019135bd89888, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:11.111973 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6283cf91-f87f-48de-a858-0466c88aa4eb admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:11.150826 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 844/1683] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:10 2022] PUT /identity/v3/projects/7231b818f44b4cffa5274ed572974963/users/2211a2bf3fa045e48ed019135bd89888/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:11.151647 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6283cf91-f87f-48de-a858-0466c88aa4eb admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:11.295115 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c407de38-abf1-4f52-aa63-0fb5922d3947 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:11.367920 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6283cf91-f87f-48de-a858-0466c88aa4eb admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:11.388020 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36744213-31e3-483f-8ed6-b3026c38ea3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:11.388020 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36744213-31e3-483f-8ed6-b3026c38ea3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:11.388020 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36744213-31e3-483f-8ed6-b3026c38ea3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:11.388020 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36744213-31e3-483f-8ed6-b3026c38ea3e admin admin] PATH_INFO: `/v3/projects/52becdc909ff44c4bf026a2ff9ee165a/users/bf706de518bd47308323f07f47039b9f/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:11.392879 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36744213-31e3-483f-8ed6-b3026c38ea3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=52becdc909ff44c4bf026a2ff9ee165a, user_id=bf706de518bd47308323f07f47039b9f, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:11.397784 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36744213-31e3-483f-8ed6-b3026c38ea3e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:11.445735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 845/1684] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:11 2022] PUT /identity/v3/projects/52becdc909ff44c4bf026a2ff9ee165a/users/bf706de518bd47308323f07f47039b9f/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:11.445735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36744213-31e3-483f-8ed6-b3026c38ea3e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:11.491544 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:11.520130 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607eea28-4e3f-4647-b317-b66a68a834e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:11.520965 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607eea28-4e3f-4647-b317-b66a68a834e6 service nova] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:11.522757 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607eea28-4e3f-4647-b317-b66a68a834e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:11.523118 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607eea28-4e3f-4647-b317-b66a68a834e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:11.712860 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36744213-31e3-483f-8ed6-b3026c38ea3e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:11.726596 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82a6807-03a2-45bf-9128-c83d78e8f01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:11.727983 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a6807-03a2-45bf-9128-c83d78e8f01a admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:11.728271 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a6807-03a2-45bf-9128-c83d78e8f01a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:11.729005 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a6807-03a2-45bf-9128-c83d78e8f01a admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:11.731454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82a6807-03a2-45bf-9128-c83d78e8f01a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:11.737729 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82a6807-03a2-45bf-9128-c83d78e8f01a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:11.751629 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607eea28-4e3f-4647-b317-b66a68a834e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:11.753538 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607eea28-4e3f-4647-b317-b66a68a834e6 service nova] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:11.770259 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 846/1685] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:11.773473 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d0726-eec1-426f-9c85-fcbe8fe482cc None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:11.773762 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d0726-eec1-426f-9c85-fcbe8fe482cc None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:11.774036 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d0726-eec1-426f-9c85-fcbe8fe482cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:11.789212 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 840/1686] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 746 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:11.791007 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607eea28-4e3f-4647-b317-b66a68a834e6 service nova] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:11.795246 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-607eea28-4e3f-4647-b317-b66a68a834e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:11.829431 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607eea28-4e3f-4647-b317-b66a68a834e6 service nova] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:11.841670 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8acbf6f-539d-42ca-91be-dc7b409be157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:11.846176 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8acbf6f-539d-42ca-91be-dc7b409be157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:11.846496 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8acbf6f-539d-42ca-91be-dc7b409be157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:11.846778 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8acbf6f-539d-42ca-91be-dc7b409be157 admin admin] PATH_INFO: `/v3/users/eb174dab2ed04d34857dcd27a17480f9` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:11.848231 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8acbf6f-539d-42ca-91be-dc7b409be157 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb174dab2ed04d34857dcd27a17480f9)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:11.850841 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8acbf6f-539d-42ca-91be-dc7b409be157 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:11.851773 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-346d0726-eec1-426f-9c85-fcbe8fe482cc None None] MFA Rules not processed for user `bf706de518bd47308323f07f47039b9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:11.853007 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-346d0726-eec1-426f-9c85-fcbe8fe482cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:11.949084 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:11.980845 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-f8acbf6f-539d-42ca-91be-dc7b409be157 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb174dab2ed04d34857dcd27a17480f9'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:12.022003 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-f8acbf6f-539d-42ca-91be-dc7b409be157 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb174dab2ed04d34857dcd27a17480f9'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:12.067739 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-f8acbf6f-539d-42ca-91be-dc7b409be157 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb174dab2ed04d34857dcd27a17480f9'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:12.087153 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 847/1687] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:11 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:12.087153 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d0726-eec1-426f-9c85-fcbe8fe482cc None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:12.094699 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-f8acbf6f-539d-42ca-91be-dc7b409be157 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb174dab2ed04d34857dcd27a17480f9'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:12.095954 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-f8acbf6f-539d-42ca-91be-dc7b409be157 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb174dab2ed04d34857dcd27a17480f9'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:12.115520 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 841/1688] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:28:11 2022] DELETE /identity/v3/users/eb174dab2ed04d34857dcd27a17480f9 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:12.119354 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3740e-d088-454a-acea-69cec3d7ca7d None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:12.123354 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3740e-d088-454a-acea-69cec3d7ca7d None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:12.123354 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3740e-d088-454a-acea-69cec3d7ca7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:12.134634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d0726-eec1-426f-9c85-fcbe8fe482cc None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:12.242289 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-3cc3740e-d088-454a-acea-69cec3d7ca7d None None] MFA Rules not processed for user `fc41cb3dbe3845d794107f068c70db3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:12.244520 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-3cc3740e-d088-454a-acea-69cec3d7ca7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:12.327350 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384583c0-8c0c-48dc-b340-bee2f1d148e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:12.329814 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384583c0-8c0c-48dc-b340-bee2f1d148e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:12.331279 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384583c0-8c0c-48dc-b340-bee2f1d148e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:12.335674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384583c0-8c0c-48dc-b340-bee2f1d148e5 admin admin] PATH_INFO: `/v3/projects/7231b818f44b4cffa5274ed572974963/users/2211a2bf3fa045e48ed019135bd89888/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:12.348436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384583c0-8c0c-48dc-b340-bee2f1d148e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7231b818f44b4cffa5274ed572974963, user_id=2211a2bf3fa045e48ed019135bd89888, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:12.354671 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384583c0-8c0c-48dc-b340-bee2f1d148e5 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:12.388700 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 848/1689] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:12 2022] PUT /identity/v3/projects/7231b818f44b4cffa5274ed572974963/users/2211a2bf3fa045e48ed019135bd89888/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:12.391870 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384583c0-8c0c-48dc-b340-bee2f1d148e5 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:12.402248 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-384583c0-8c0c-48dc-b340-bee2f1d148e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:12.418638 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 842/1690] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:12.420622 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f030d5-6709-416e-a2c4-2407efa99249 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:12.426833 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f030d5-6709-416e-a2c4-2407efa99249 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:12.427349 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f030d5-6709-416e-a2c4-2407efa99249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:12.457936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-21f030d5-6709-416e-a2c4-2407efa99249 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:12.461100 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-21f030d5-6709-416e-a2c4-2407efa99249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:12.484318 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:12.534241 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384583c0-8c0c-48dc-b340-bee2f1d148e5 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:12.544328 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e36d3-c52b-42d9-8b39-46bead482ab4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:12.545630 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e36d3-c52b-42d9-8b39-46bead482ab4 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:12.546665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e36d3-c52b-42d9-8b39-46bead482ab4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:12.547363 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e36d3-c52b-42d9-8b39-46bead482ab4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:12.554466 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5a9e36d3-c52b-42d9-8b39-46bead482ab4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:12.654466 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 843/1691] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:12.662325 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea16a4-a893-48b4-92ce-6bcc20cb30e9 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:12.664239 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea16a4-a893-48b4-92ce-6bcc20cb30e9 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:12.664603 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea16a4-a893-48b4-92ce-6bcc20cb30e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:12.733076 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-bdea16a4-a893-48b4-92ce-6bcc20cb30e9 None None] MFA Rules not processed for user `2211a2bf3fa045e48ed019135bd89888`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:12.735593 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-bdea16a4-a893-48b4-92ce-6bcc20cb30e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:12.743577 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9e36d3-c52b-42d9-8b39-46bead482ab4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:12.747474 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9e36d3-c52b-42d9-8b39-46bead482ab4 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:12.777535 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 849/1692] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:12 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:12.781399 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d6e47-d019-49cb-af05-146d44e5ae0b None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:12.781701 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d6e47-d019-49cb-af05-146d44e5ae0b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:12.782990 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d6e47-d019-49cb-af05-146d44e5ae0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:12.831450 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:12.854341 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-860d6e47-d019-49cb-af05-146d44e5ae0b None None] MFA Rules not processed for user `fc41cb3dbe3845d794107f068c70db3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:12.856315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-860d6e47-d019-49cb-af05-146d44e5ae0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:12.944090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 844/1693] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:12 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:12.947051 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea16a4-a893-48b4-92ce-6bcc20cb30e9 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:12.954181 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-bdea16a4-a893-48b4-92ce-6bcc20cb30e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:13.026838 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 850/1694] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:13.026838 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d6e47-d019-49cb-af05-146d44e5ae0b None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:13.057093 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d6e47-d019-49cb-af05-146d44e5ae0b None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:13.070649 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507744f-bf4d-4343-b25a-e5edee98e7e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:13.071427 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507744f-bf4d-4343-b25a-e5edee98e7e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:13.071813 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507744f-bf4d-4343-b25a-e5edee98e7e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:13.072187 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507744f-bf4d-4343-b25a-e5edee98e7e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:13.096972 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507744f-bf4d-4343-b25a-e5edee98e7e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:13.099728 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507744f-bf4d-4343-b25a-e5edee98e7e4 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:13.111842 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea16a4-a893-48b4-92ce-6bcc20cb30e9 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:13.121588 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694e302-5efe-4e55-a448-7d387b64301b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:13.122377 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694e302-5efe-4e55-a448-7d387b64301b service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:13.122750 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694e302-5efe-4e55-a448-7d387b64301b service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:13.123387 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694e302-5efe-4e55-a448-7d387b64301b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:13.139262 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 851/1695] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:13 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:13.142038 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507744f-bf4d-4343-b25a-e5edee98e7e4 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:13.146146 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1507744f-bf4d-4343-b25a-e5edee98e7e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:13.165640 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6694e302-5efe-4e55-a448-7d387b64301b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:13.170774 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6694e302-5efe-4e55-a448-7d387b64301b service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:13.225959 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 845/1696] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:13.229485 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694e302-5efe-4e55-a448-7d387b64301b service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:13.234496 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6694e302-5efe-4e55-a448-7d387b64301b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:13.285416 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694e302-5efe-4e55-a448-7d387b64301b service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:13.310267 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7573f2b-757f-4a5b-b577-93cc5bdb062a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:13.310937 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7573f2b-757f-4a5b-b577-93cc5bdb062a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:13.311219 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7573f2b-757f-4a5b-b577-93cc5bdb062a admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:13.311497 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7573f2b-757f-4a5b-b577-93cc5bdb062a admin admin] PATH_INFO: `/v3/users/0b6c61dc11e04ea8b3492287d52b615a` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:13.318068 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7573f2b-757f-4a5b-b577-93cc5bdb062a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b6c61dc11e04ea8b3492287d52b615a)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:13.323501 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7573f2b-757f-4a5b-b577-93cc5bdb062a admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:13.416801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507744f-bf4d-4343-b25a-e5edee98e7e4 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:13.434121 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584c317-cc33-464f-849d-9807f1b0fbb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:13.434717 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584c317-cc33-464f-849d-9807f1b0fbb0 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:13.435242 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584c317-cc33-464f-849d-9807f1b0fbb0 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:13.435629 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584c317-cc33-464f-849d-9807f1b0fbb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:13.457562 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:13.462045 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3584c317-cc33-464f-849d-9807f1b0fbb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:13.462827 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-b7573f2b-757f-4a5b-b577-93cc5bdb062a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b6c61dc11e04ea8b3492287d52b615a'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:13.464521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3584c317-cc33-464f-849d-9807f1b0fbb0 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:13.475851 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 852/1697] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:13 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:13.504704 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-b7573f2b-757f-4a5b-b577-93cc5bdb062a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b6c61dc11e04ea8b3492287d52b615a'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:13.518541 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-b7573f2b-757f-4a5b-b577-93cc5bdb062a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b6c61dc11e04ea8b3492287d52b615a'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:13.532757 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-b7573f2b-757f-4a5b-b577-93cc5bdb062a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b6c61dc11e04ea8b3492287d52b615a'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:13.533879 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-b7573f2b-757f-4a5b-b577-93cc5bdb062a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b6c61dc11e04ea8b3492287d52b615a'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:13.557332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 846/1698] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:28:13 2022] DELETE /identity/v3/users/0b6c61dc11e04ea8b3492287d52b615a => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:13.559081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc465559-c60a-421e-bcc4-8332198dc9c3 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:13.559490 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc465559-c60a-421e-bcc4-8332198dc9c3 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:13.559873 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc465559-c60a-421e-bcc4-8332198dc9c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:13.618250 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-bc465559-c60a-421e-bcc4-8332198dc9c3 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:13.622079 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-bc465559-c60a-421e-bcc4-8332198dc9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:13.817729 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 853/1699] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:13.879383 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7573f2b-757f-4a5b-b577-93cc5bdb062a admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:13.881150 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-b7573f2b-757f-4a5b-b577-93cc5bdb062a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:14.048063 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc465559-c60a-421e-bcc4-8332198dc9c3 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:14.049811 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-bc465559-c60a-421e-bcc4-8332198dc9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:14.075595 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc465559-c60a-421e-bcc4-8332198dc9c3 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:14.086276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e3febe-5c3b-44b8-bb21-4fc120ad1b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:14.089173 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e3febe-5c3b-44b8-bb21-4fc120ad1b9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:14.090749 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e3febe-5c3b-44b8-bb21-4fc120ad1b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:14.091053 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e3febe-5c3b-44b8-bb21-4fc120ad1b9d admin admin] PATH_INFO: `/v3/users/23134965e73a4d4a89e8853f9d49ecab` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:14.095378 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e3febe-5c3b-44b8-bb21-4fc120ad1b9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=23134965e73a4d4a89e8853f9d49ecab)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:14.100537 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e3febe-5c3b-44b8-bb21-4fc120ad1b9d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:14.116967 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:14.171315 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7573f2b-757f-4a5b-b577-93cc5bdb062a admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:14.183303 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8c2d8-c418-4e77-898b-f875d59ccd9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:14.183950 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8c2d8-c418-4e77-898b-f875d59ccd9d service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:14.184231 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8c2d8-c418-4e77-898b-f875d59ccd9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:14.184513 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8c2d8-c418-4e77-898b-f875d59ccd9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:14.213730 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-44e3febe-5c3b-44b8-bb21-4fc120ad1b9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23134965e73a4d4a89e8853f9d49ecab'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:14.223476 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-44e3febe-5c3b-44b8-bb21-4fc120ad1b9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23134965e73a4d4a89e8853f9d49ecab'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:14.234297 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a8c2d8-c418-4e77-898b-f875d59ccd9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:14.242343 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-44e3febe-5c3b-44b8-bb21-4fc120ad1b9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23134965e73a4d4a89e8853f9d49ecab'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:14.244760 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a8c2d8-c418-4e77-898b-f875d59ccd9d service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:14.252051 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-44e3febe-5c3b-44b8-bb21-4fc120ad1b9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23134965e73a4d4a89e8853f9d49ecab'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:14.252820 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-44e3febe-5c3b-44b8-bb21-4fc120ad1b9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23134965e73a4d4a89e8853f9d49ecab'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:14.270268 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 854/1700] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:28:14 2022] DELETE /identity/v3/users/23134965e73a4d4a89e8853f9d49ecab => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:14.275378 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4976d64-59e9-4ee1-b5e2-abfba2b145bd None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:14.275869 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4976d64-59e9-4ee1-b5e2-abfba2b145bd None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:14.276251 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4976d64-59e9-4ee1-b5e2-abfba2b145bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:14.281734 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 847/1701] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:14.314038 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f4976d64-59e9-4ee1-b5e2-abfba2b145bd None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:14.315727 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f4976d64-59e9-4ee1-b5e2-abfba2b145bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:14.432215 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 855/1702] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:14.516669 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8c2d8-c418-4e77-898b-f875d59ccd9d service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:14.530134 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8c2d8-c418-4e77-898b-f875d59ccd9d service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:14.694554 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4293cf-b960-4518-8437-d6eaaa691713 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:14.695365 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4293cf-b960-4518-8437-d6eaaa691713 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:14.695656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4293cf-b960-4518-8437-d6eaaa691713 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:14.695933 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4293cf-b960-4518-8437-d6eaaa691713 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:14.721813 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4293cf-b960-4518-8437-d6eaaa691713 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:14.726685 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4293cf-b960-4518-8437-d6eaaa691713 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:14.758591 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 848/1703] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:15.042776 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9c7ef-8355-49fc-ae87-4738224db505 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:15.042776 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9c7ef-8355-49fc-ae87-4738224db505 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:15.043905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9c7ef-8355-49fc-ae87-4738224db505 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:15.046094 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 856/1704] 10.222.0.26 () {54 vars in 912 bytes} [Fri Sep 9 20:28:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:28:15.051911 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9022e76-cdf6-4875-b718-69fb140aad80 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:15.052247 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9022e76-cdf6-4875-b718-69fb140aad80 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:15.052451 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9022e76-cdf6-4875-b718-69fb140aad80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:15.161924 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-f9022e76-cdf6-4875-b718-69fb140aad80 None None] MFA Rules not processed for user `ee5c6e51f2694bf5ae7ea097cf33fb3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:15.163929 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-f9022e76-cdf6-4875-b718-69fb140aad80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:15.182762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 849/1705] 10.222.0.26 () {58 vars in 1009 bytes} [Fri Sep 9 20:28:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:15.200934 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9c7ef-8355-49fc-ae87-4738224db505 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:15.206625 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-64c9c7ef-8355-49fc-ae87-4738224db505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:15.241556 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9c7ef-8355-49fc-ae87-4738224db505 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:15.244158 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9022e76-cdf6-4875-b718-69fb140aad80 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:15.246815 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-f9022e76-cdf6-4875-b718-69fb140aad80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:15.247106 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73525b38-762e-4286-a206-3fbef22650d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:15.247904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73525b38-762e-4286-a206-3fbef22650d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:15.248767 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73525b38-762e-4286-a206-3fbef22650d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:15.255482 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73525b38-762e-4286-a206-3fbef22650d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:15.261544 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:15.270014 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73525b38-762e-4286-a206-3fbef22650d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:15.276017 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73525b38-762e-4286-a206-3fbef22650d7 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:15.279235 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9022e76-cdf6-4875-b718-69fb140aad80 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:15.289988 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d8a037-b061-44a5-9612-9e89c7022608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:15.290856 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8a037-b061-44a5-9612-9e89c7022608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:15.291243 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8a037-b061-44a5-9612-9e89c7022608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:15.291812 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8a037-b061-44a5-9612-9e89c7022608 admin admin] PATH_INFO: `/v3/projects/3bae95c402e44346959aca56968761aa` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:15.292094 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 857/1706] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:15.293196 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d8a037-b061-44a5-9612-9e89c7022608 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bae95c402e44346959aca56968761aa)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:15.296401 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d8a037-b061-44a5-9612-9e89c7022608 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:15.381401 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-28d8a037-b061-44a5-9612-9e89c7022608 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bae95c402e44346959aca56968761aa', 'request_id': 'req-28d8a037-b061-44a5-9612-9e89c7022608'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:15.392059 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-28d8a037-b061-44a5-9612-9e89c7022608 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bae95c402e44346959aca56968761aa', 'request_id': 'req-28d8a037-b061-44a5-9612-9e89c7022608'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:15.400302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-28d8a037-b061-44a5-9612-9e89c7022608 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bae95c402e44346959aca56968761aa', 'request_id': 'req-28d8a037-b061-44a5-9612-9e89c7022608'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:15.416128 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-28d8a037-b061-44a5-9612-9e89c7022608 admin admin] The token cache is being invalidate because project 3bae95c402e44346959aca56968761aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:28:15.417321 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-28d8a037-b061-44a5-9612-9e89c7022608 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:15.418392 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-28d8a037-b061-44a5-9612-9e89c7022608 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:15.433776 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 850/1707] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:28:15 2022] DELETE /identity/v3/projects/3bae95c402e44346959aca56968761aa => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:15.439552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56e8af-4c2a-4b01-b063-7758b05845d4 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:15.440047 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56e8af-4c2a-4b01-b063-7758b05845d4 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:15.440382 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56e8af-4c2a-4b01-b063-7758b05845d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:15.459586 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d8a037-b061-44a5-9612-9e89c7022608 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:15.462032 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-28d8a037-b061-44a5-9612-9e89c7022608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:15.511005 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-6b56e8af-4c2a-4b01-b063-7758b05845d4 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:15.512622 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6b56e8af-4c2a-4b01-b063-7758b05845d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:15.580767 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:15.620596 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d8a037-b061-44a5-9612-9e89c7022608 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:15.634606 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09554daa-87f4-4814-a406-ee2184a8b37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:15.635440 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09554daa-87f4-4814-a406-ee2184a8b37b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:15.635854 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09554daa-87f4-4814-a406-ee2184a8b37b admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:15.636237 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09554daa-87f4-4814-a406-ee2184a8b37b admin admin] PATH_INFO: `/v3/users/9d15eb60e90f44aab02044a61e7aa461` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:15.637704 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09554daa-87f4-4814-a406-ee2184a8b37b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d15eb60e90f44aab02044a61e7aa461)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:15.639961 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09554daa-87f4-4814-a406-ee2184a8b37b admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:15.676166 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 858/1708] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:15.683855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef699c9d-c033-42de-97f4-54ed602a25ec None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:15.684146 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef699c9d-c033-42de-97f4-54ed602a25ec None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:15.684468 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef699c9d-c033-42de-97f4-54ed602a25ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:15.726441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ef699c9d-c033-42de-97f4-54ed602a25ec None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:15.727725 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ef699c9d-c033-42de-97f4-54ed602a25ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:15.738390 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-09554daa-87f4-4814-a406-ee2184a8b37b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d15eb60e90f44aab02044a61e7aa461'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:15.753572 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 859/1709] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:28:15.757614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef699c9d-c033-42de-97f4-54ed602a25ec None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:15.770441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef699c9d-c033-42de-97f4-54ed602a25ec None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:15.786047 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9958b-e7e7-4f7f-80d4-7c284ac0f33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:15.786679 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9958b-e7e7-4f7f-80d4-7c284ac0f33c admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:15.786956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9958b-e7e7-4f7f-80d4-7c284ac0f33c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:15.787225 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9958b-e7e7-4f7f-80d4-7c284ac0f33c admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:15.787881 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9958b-e7e7-4f7f-80d4-7c284ac0f33c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:28:15.788178 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9958b-e7e7-4f7f-80d4-7c284ac0f33c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:15.789855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9958b-e7e7-4f7f-80d4-7c284ac0f33c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:15.792329 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-09554daa-87f4-4814-a406-ee2184a8b37b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d15eb60e90f44aab02044a61e7aa461'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:15.801515 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-09554daa-87f4-4814-a406-ee2184a8b37b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d15eb60e90f44aab02044a61e7aa461'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:15.818052 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-09554daa-87f4-4814-a406-ee2184a8b37b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d15eb60e90f44aab02044a61e7aa461'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:15.819862 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-09554daa-87f4-4814-a406-ee2184a8b37b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d15eb60e90f44aab02044a61e7aa461'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:15.821952 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 860/1710] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:28:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:28:15.826737 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9958b-e7e7-4f7f-80d4-7c284ac0f33c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:15.828685 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7df9958b-e7e7-4f7f-80d4-7c284ac0f33c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:15.849682 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 851/1711] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:28:15 2022] DELETE /identity/v3/users/9d15eb60e90f44aab02044a61e7aa461 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:15.857129 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c53d7-3fe4-4893-bc3b-bd0e28c165bd None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:15.858208 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c53d7-3fe4-4893-bc3b-bd0e28c165bd None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:15.859803 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c53d7-3fe4-4893-bc3b-bd0e28c165bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:15.862018 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9958b-e7e7-4f7f-80d4-7c284ac0f33c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:15.904402 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-976c53d7-3fe4-4893-bc3b-bd0e28c165bd None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:15.908527 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-976c53d7-3fe4-4893-bc3b-bd0e28c165bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:16.015883 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c9b10f-a37e-432c-996d-7c036bbef77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:16.016644 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c9b10f-a37e-432c-996d-7c036bbef77a admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:16.017018 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c9b10f-a37e-432c-996d-7c036bbef77a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:16.017420 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c9b10f-a37e-432c-996d-7c036bbef77a admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:16.018167 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c9b10f-a37e-432c-996d-7c036bbef77a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:16.022374 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c9b10f-a37e-432c-996d-7c036bbef77a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:16.068439 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 852/1712] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:16.092167 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976c53d7-3fe4-4893-bc3b-bd0e28c165bd None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:16.095785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-976c53d7-3fe4-4893-bc3b-bd0e28c165bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:16.104130 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 861/1713] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:28:15 2022] POST /identity/v3/projects => generated 375 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:16.105074 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c9b10f-a37e-432c-996d-7c036bbef77a admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:16.112078 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-83c9b10f-a37e-432c-996d-7c036bbef77a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:16.129709 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976c53d7-3fe4-4893-bc3b-bd0e28c165bd None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:16.236326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442a1b59-1425-4e4f-86a2-7709f1710674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:16.238198 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442a1b59-1425-4e4f-86a2-7709f1710674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:16.238644 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442a1b59-1425-4e4f-86a2-7709f1710674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:16.239070 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442a1b59-1425-4e4f-86a2-7709f1710674 admin admin] PATH_INFO: `/v3/projects/d15626d86387448eb1d19803feb3851d` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:16.240914 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442a1b59-1425-4e4f-86a2-7709f1710674 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d15626d86387448eb1d19803feb3851d)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:16.244037 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442a1b59-1425-4e4f-86a2-7709f1710674 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:16.260907 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:16.314232 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c9b10f-a37e-432c-996d-7c036bbef77a admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:16.339068 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d446e5-0bec-4f42-915f-21171a7b4db7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:16.339068 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d446e5-0bec-4f42-915f-21171a7b4db7 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:16.339457 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d446e5-0bec-4f42-915f-21171a7b4db7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:16.339657 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d446e5-0bec-4f42-915f-21171a7b4db7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:16.363822 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d446e5-0bec-4f42-915f-21171a7b4db7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:16.365305 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d446e5-0bec-4f42-915f-21171a7b4db7 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:16.385156 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-442a1b59-1425-4e4f-86a2-7709f1710674 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd15626d86387448eb1d19803feb3851d', 'request_id': 'req-442a1b59-1425-4e4f-86a2-7709f1710674'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:16.396489 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 862/1714] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:16.398632 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d446e5-0bec-4f42-915f-21171a7b4db7 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:16.404748 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:16.416449 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-442a1b59-1425-4e4f-86a2-7709f1710674 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd15626d86387448eb1d19803feb3851d', 'request_id': 'req-442a1b59-1425-4e4f-86a2-7709f1710674'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:16.421160 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d446e5-0bec-4f42-915f-21171a7b4db7 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:16.432053 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-442a1b59-1425-4e4f-86a2-7709f1710674 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd15626d86387448eb1d19803feb3851d', 'request_id': 'req-442a1b59-1425-4e4f-86a2-7709f1710674'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:16.461159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-442a1b59-1425-4e4f-86a2-7709f1710674 admin admin] The token cache is being invalidate because project d15626d86387448eb1d19803feb3851d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:28:16.463583 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-442a1b59-1425-4e4f-86a2-7709f1710674 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:16.465567 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-442a1b59-1425-4e4f-86a2-7709f1710674 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:16.470728 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 853/1715] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:28:16 2022] DELETE /identity/v3/projects/d15626d86387448eb1d19803feb3851d => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:16.472254 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442a1b59-1425-4e4f-86a2-7709f1710674 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:16.490119 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-442a1b59-1425-4e4f-86a2-7709f1710674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:16.613029 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169beb4-1ece-492c-9368-4eccb539e814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:16.616581 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169beb4-1ece-492c-9368-4eccb539e814 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:16.619783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169beb4-1ece-492c-9368-4eccb539e814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:16.620990 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169beb4-1ece-492c-9368-4eccb539e814 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:16.621839 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d169beb4-1ece-492c-9368-4eccb539e814 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:16.627252 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d169beb4-1ece-492c-9368-4eccb539e814 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:16.695857 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 863/1716] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:28:16 2022] POST /identity/v3/users => generated 288 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:16.698965 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169beb4-1ece-492c-9368-4eccb539e814 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:16.714919 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d169beb4-1ece-492c-9368-4eccb539e814 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:16.746280 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169beb4-1ece-492c-9368-4eccb539e814 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:16.747397 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442a1b59-1425-4e4f-86a2-7709f1710674 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:16.761901 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a60c0-a9ea-4d6a-a3cc-50b44945f149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:16.764247 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a60c0-a9ea-4d6a-a3cc-50b44945f149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:16.765336 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfba9c7-c24f-49d3-bf2a-798af1386cc8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26d24cf0a21d491082b0c2f11e749446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:16.765863 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a60c0-a9ea-4d6a-a3cc-50b44945f149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:16.766731 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfba9c7-c24f-49d3-bf2a-798af1386cc8 service placement] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:16.767217 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a60c0-a9ea-4d6a-a3cc-50b44945f149 admin admin] PATH_INFO: `/v3/users/2211a2bf3fa045e48ed019135bd89888` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:16.768882 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfba9c7-c24f-49d3-bf2a-798af1386cc8 service placement] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:16.770227 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8a60c0-a9ea-4d6a-a3cc-50b44945f149 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2211a2bf3fa045e48ed019135bd89888)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:16.770610 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfba9c7-c24f-49d3-bf2a-798af1386cc8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:16.773945 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8a60c0-a9ea-4d6a-a3cc-50b44945f149 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:16.778368 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-4dfba9c7-c24f-49d3-bf2a-798af1386cc8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:16.806162 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfba9c7-c24f-49d3-bf2a-798af1386cc8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:16.818091 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfba9c7-c24f-49d3-bf2a-798af1386cc8 service placement] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:16.859979 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 854/1717] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:16.861181 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfba9c7-c24f-49d3-bf2a-798af1386cc8 service placement] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:16.865434 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-4dfba9c7-c24f-49d3-bf2a-798af1386cc8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:16.918220 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfba9c7-c24f-49d3-bf2a-798af1386cc8 service placement] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:16.940715 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1fe034-0edf-45cc-a212-adbb2b046ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:16.941384 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1fe034-0edf-45cc-a212-adbb2b046ff5 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:16.941672 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1fe034-0edf-45cc-a212-adbb2b046ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:16.941950 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1fe034-0edf-45cc-a212-adbb2b046ff5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:16.942613 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1fe034-0edf-45cc-a212-adbb2b046ff5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:16.944228 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1fe034-0edf-45cc-a212-adbb2b046ff5 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:16.945404 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6f8a60c0-a9ea-4d6a-a3cc-50b44945f149 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2211a2bf3fa045e48ed019135bd89888'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:16.964533 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6f8a60c0-a9ea-4d6a-a3cc-50b44945f149 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2211a2bf3fa045e48ed019135bd89888'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:16.979998 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 855/1718] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:16.983328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1fe034-0edf-45cc-a212-adbb2b046ff5 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:17.000683 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6f8a60c0-a9ea-4d6a-a3cc-50b44945f149 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2211a2bf3fa045e48ed019135bd89888'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:17.011121 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1fe034-0edf-45cc-a212-adbb2b046ff5 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:17.017981 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dceaf9-3b5a-4376-bb70-5d5fe10ff829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:17.019361 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dceaf9-3b5a-4376-bb70-5d5fe10ff829 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:17.019706 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dceaf9-3b5a-4376-bb70-5d5fe10ff829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:17.020045 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dceaf9-3b5a-4376-bb70-5d5fe10ff829 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:17.021554 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6f8a60c0-a9ea-4d6a-a3cc-50b44945f149 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2211a2bf3fa045e48ed019135bd89888'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:17.022221 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6f8a60c0-a9ea-4d6a-a3cc-50b44945f149 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2211a2bf3fa045e48ed019135bd89888'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:17.029997 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dceaf9-3b5a-4376-bb70-5d5fe10ff829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:17.032481 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 864/1719] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:28:16 2022] DELETE /identity/v3/users/2211a2bf3fa045e48ed019135bd89888 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:17.042441 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dceaf9-3b5a-4376-bb70-5d5fe10ff829 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:17.061288 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 856/1720] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:17.065480 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a60c0-a9ea-4d6a-a3cc-50b44945f149 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:17.067859 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6f8a60c0-a9ea-4d6a-a3cc-50b44945f149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:17.188617 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a60c0-a9ea-4d6a-a3cc-50b44945f149 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:17.194545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408d12a-596e-4171-9353-a0f1e26692f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:17.195901 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408d12a-596e-4171-9353-a0f1e26692f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:17.196265 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408d12a-596e-4171-9353-a0f1e26692f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:17.196614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408d12a-596e-4171-9353-a0f1e26692f9 admin admin] PATH_INFO: `/v3/projects/99832d5ba9b7408b88d9c0f022cd6ef2/users/65a1c557deb8457ba38878670b96e56d/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:17.211640 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:17.232134 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e408d12a-596e-4171-9353-a0f1e26692f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99832d5ba9b7408b88d9c0f022cd6ef2, user_id=65a1c557deb8457ba38878670b96e56d, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:17.241313 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e408d12a-596e-4171-9353-a0f1e26692f9 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:17.289074 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 865/1721] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:17 2022] PUT /identity/v3/projects/99832d5ba9b7408b88d9c0f022cd6ef2/users/65a1c557deb8457ba38878670b96e56d/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:28:17.292495 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16feac88-7047-487a-93d3-d0963c352e88 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:17.292966 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16feac88-7047-487a-93d3-d0963c352e88 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:17.293051 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16feac88-7047-487a-93d3-d0963c352e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:17.359828 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-16feac88-7047-487a-93d3-d0963c352e88 None None] MFA Rules not processed for user `65a1c557deb8457ba38878670b96e56d`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:17.361209 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-16feac88-7047-487a-93d3-d0963c352e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:17.534121 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408d12a-596e-4171-9353-a0f1e26692f9 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:17.535946 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e408d12a-596e-4171-9353-a0f1e26692f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:17.556658 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:17.563704 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 857/1722] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:17.568841 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16feac88-7047-487a-93d3-d0963c352e88 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:17.719314 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408d12a-596e-4171-9353-a0f1e26692f9 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:17.726507 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d1842-0b4f-4a2c-bf4b-1795755ddd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:17.727549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d1842-0b4f-4a2c-bf4b-1795755ddd16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:17.728037 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d1842-0b4f-4a2c-bf4b-1795755ddd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:17.728538 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d1842-0b4f-4a2c-bf4b-1795755ddd16 admin admin] PATH_INFO: `/v3/projects/4a7c343e5c21424099577ade88f2f903` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:17.736667 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061d1842-0b4f-4a2c-bf4b-1795755ddd16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a7c343e5c21424099577ade88f2f903)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:17.739146 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061d1842-0b4f-4a2c-bf4b-1795755ddd16 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:17.748213 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16feac88-7047-487a-93d3-d0963c352e88 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:17.755748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21673ab2-21fc-45c0-9dbf-03dd4ef8e714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:17.756813 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21673ab2-21fc-45c0-9dbf-03dd4ef8e714 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:17.757445 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21673ab2-21fc-45c0-9dbf-03dd4ef8e714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:17.757828 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21673ab2-21fc-45c0-9dbf-03dd4ef8e714 admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:17.758676 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21673ab2-21fc-45c0-9dbf-03dd4ef8e714 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:17.762553 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21673ab2-21fc-45c0-9dbf-03dd4ef8e714 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:17.839463 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 858/1723] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:28:17 2022] POST /identity/v3/projects => generated 379 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:28:17.841072 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a14ee-40ad-4a1c-b639-cc4f1e8ecf55 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:17.841072 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a14ee-40ad-4a1c-b639-cc4f1e8ecf55 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:17.841306 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a14ee-40ad-4a1c-b639-cc4f1e8ecf55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:17.844178 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-061d1842-0b4f-4a2c-bf4b-1795755ddd16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a7c343e5c21424099577ade88f2f903', 'request_id': 'req-061d1842-0b4f-4a2c-bf4b-1795755ddd16'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:17.873305 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-061d1842-0b4f-4a2c-bf4b-1795755ddd16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a7c343e5c21424099577ade88f2f903', 'request_id': 'req-061d1842-0b4f-4a2c-bf4b-1795755ddd16'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:17.894740 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-061d1842-0b4f-4a2c-bf4b-1795755ddd16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a7c343e5c21424099577ade88f2f903', 'request_id': 'req-061d1842-0b4f-4a2c-bf4b-1795755ddd16'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:17.909763 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-a93a14ee-40ad-4a1c-b639-cc4f1e8ecf55 None None] MFA Rules not processed for user `db1f365ebf794941b3999adb37c48100`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:17.913259 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a93a14ee-40ad-4a1c-b639-cc4f1e8ecf55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:17.920358 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-061d1842-0b4f-4a2c-bf4b-1795755ddd16 admin admin] The token cache is being invalidate because project 4a7c343e5c21424099577ade88f2f903 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:28:17.921486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-061d1842-0b4f-4a2c-bf4b-1795755ddd16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:17.923418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-061d1842-0b4f-4a2c-bf4b-1795755ddd16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:17.927916 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 866/1724] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:28:17 2022] DELETE /identity/v3/projects/4a7c343e5c21424099577ade88f2f903 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:28:17.932722 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90aa20-970b-4ad1-b697-2425c753a9ed None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:17.933075 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90aa20-970b-4ad1-b697-2425c753a9ed None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:17.933385 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90aa20-970b-4ad1-b697-2425c753a9ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:17.986338 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-6a90aa20-970b-4ad1-b697-2425c753a9ed None None] MFA Rules not processed for user `95bc48b6541c4388b45435036a1c4525`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:17.987581 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6a90aa20-970b-4ad1-b697-2425c753a9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:18.084190 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 859/1725] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:17 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:18.087294 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a14ee-40ad-4a1c-b639-cc4f1e8ecf55 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:18.092832 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a93a14ee-40ad-4a1c-b639-cc4f1e8ecf55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:18.137046 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 867/1726] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:18.138127 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90aa20-970b-4ad1-b697-2425c753a9ed None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:18.156182 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6a90aa20-970b-4ad1-b697-2425c753a9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:18.207708 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:18.243962 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a14ee-40ad-4a1c-b639-cc4f1e8ecf55 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:18.254236 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc5db7c-c09c-471b-b2b4-8649a548e7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:18.258256 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5db7c-c09c-471b-b2b4-8649a548e7af admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:18.258541 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5db7c-c09c-471b-b2b4-8649a548e7af admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:18.258811 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5db7c-c09c-471b-b2b4-8649a548e7af admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:18.260888 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc5db7c-c09c-471b-b2b4-8649a548e7af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:18.263757 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc5db7c-c09c-471b-b2b4-8649a548e7af admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:18.306677 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 860/1727] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:28:18 2022] POST /identity/v3/users => generated 296 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:18.307918 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc5db7c-c09c-471b-b2b4-8649a548e7af admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:18.312779 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90aa20-970b-4ad1-b697-2425c753a9ed None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:18.315436 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7bc5db7c-c09c-471b-b2b4-8649a548e7af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:18.331253 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:18.334461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fe2a09-99ef-483c-bdef-dd1404760ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:18.335247 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe2a09-99ef-483c-bdef-dd1404760ea8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:18.339019 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe2a09-99ef-483c-bdef-dd1404760ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:18.339513 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe2a09-99ef-483c-bdef-dd1404760ea8 admin admin] PATH_INFO: `/v3/users/bf706de518bd47308323f07f47039b9f` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:18.343027 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fe2a09-99ef-483c-bdef-dd1404760ea8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf706de518bd47308323f07f47039b9f)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:18.344627 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fe2a09-99ef-483c-bdef-dd1404760ea8 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:18.459053 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-10fe2a09-99ef-483c-bdef-dd1404760ea8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf706de518bd47308323f07f47039b9f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:18.478106 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-10fe2a09-99ef-483c-bdef-dd1404760ea8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf706de518bd47308323f07f47039b9f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:18.520220 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc5db7c-c09c-471b-b2b4-8649a548e7af admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:18.524066 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-10fe2a09-99ef-483c-bdef-dd1404760ea8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf706de518bd47308323f07f47039b9f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:18.537313 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f55fd5-0caf-4d2f-9cec-6b6974cb6ff9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:18.538013 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f55fd5-0caf-4d2f-9cec-6b6974cb6ff9 service nova] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:18.538013 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f55fd5-0caf-4d2f-9cec-6b6974cb6ff9 service nova] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:18.538313 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f55fd5-0caf-4d2f-9cec-6b6974cb6ff9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:18.541635 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-10fe2a09-99ef-483c-bdef-dd1404760ea8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf706de518bd47308323f07f47039b9f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:18.543933 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-10fe2a09-99ef-483c-bdef-dd1404760ea8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf706de518bd47308323f07f47039b9f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:18.565355 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 868/1728] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:28:18 2022] DELETE /identity/v3/users/bf706de518bd47308323f07f47039b9f => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:18.566662 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727461e1-be13-4a8c-9142-ff4526b3ef6d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:18.566957 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727461e1-be13-4a8c-9142-ff4526b3ef6d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:18.567235 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727461e1-be13-4a8c-9142-ff4526b3ef6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:18.571821 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f55fd5-0caf-4d2f-9cec-6b6974cb6ff9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:18.578995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f55fd5-0caf-4d2f-9cec-6b6974cb6ff9 service nova] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:18.620129 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 861/1729] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:18 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:18.621106 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f55fd5-0caf-4d2f-9cec-6b6974cb6ff9 service nova] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:18.627818 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-e3f55fd5-0caf-4d2f-9cec-6b6974cb6ff9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:18.675486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-727461e1-be13-4a8c-9142-ff4526b3ef6d None None] MFA Rules not processed for user `a95746e7a84541199b43138b1a348386`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:18.677391 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-727461e1-be13-4a8c-9142-ff4526b3ef6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:18.686201 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f55fd5-0caf-4d2f-9cec-6b6974cb6ff9 service nova] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:18.846922 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1abca-2dce-44c3-a80c-90a0576c1309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:18.847266 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1abca-2dce-44c3-a80c-90a0576c1309 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:18.847827 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1abca-2dce-44c3-a80c-90a0576c1309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:18.848171 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1abca-2dce-44c3-a80c-90a0576c1309 admin admin] PATH_INFO: `/v3/projects/afe2f4380cb24459929acfd48d2c40ad` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:18.850010 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a1abca-2dce-44c3-a80c-90a0576c1309 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afe2f4380cb24459929acfd48d2c40ad)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:18.852678 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a1abca-2dce-44c3-a80c-90a0576c1309 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:18.886748 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 869/1730] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:18 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:18.888879 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727461e1-be13-4a8c-9142-ff4526b3ef6d None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:18.893831 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-727461e1-be13-4a8c-9142-ff4526b3ef6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:18.937769 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-d2a1abca-2dce-44c3-a80c-90a0576c1309 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afe2f4380cb24459929acfd48d2c40ad', 'request_id': 'req-d2a1abca-2dce-44c3-a80c-90a0576c1309'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:18.948417 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727461e1-be13-4a8c-9142-ff4526b3ef6d None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:18.965760 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-d2a1abca-2dce-44c3-a80c-90a0576c1309 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afe2f4380cb24459929acfd48d2c40ad', 'request_id': 'req-d2a1abca-2dce-44c3-a80c-90a0576c1309'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:18.994132 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-d2a1abca-2dce-44c3-a80c-90a0576c1309 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afe2f4380cb24459929acfd48d2c40ad', 'request_id': 'req-d2a1abca-2dce-44c3-a80c-90a0576c1309'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:19.026009 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-d2a1abca-2dce-44c3-a80c-90a0576c1309 admin admin] The token cache is being invalidate because project afe2f4380cb24459929acfd48d2c40ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:28:19.026563 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-d2a1abca-2dce-44c3-a80c-90a0576c1309 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:19.027655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-d2a1abca-2dce-44c3-a80c-90a0576c1309 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:19.029872 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 862/1731] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:28:18 2022] DELETE /identity/v3/projects/afe2f4380cb24459929acfd48d2c40ad => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:19.035398 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1abca-2dce-44c3-a80c-90a0576c1309 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:19.037294 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-d2a1abca-2dce-44c3-a80c-90a0576c1309 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:19.069765 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:19.092430 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:19.156604 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1abca-2dce-44c3-a80c-90a0576c1309 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:19.169671 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d84b0-3ed4-464a-a12b-d4625106ee8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:19.173093 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d84b0-3ed4-464a-a12b-d4625106ee8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:19.175086 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d84b0-3ed4-464a-a12b-d4625106ee8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:19.175583 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d84b0-3ed4-464a-a12b-d4625106ee8c admin admin] PATH_INFO: `/v3/projects/7231b818f44b4cffa5274ed572974963` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:19.177007 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d84b0-3ed4-464a-a12b-d4625106ee8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7231b818f44b4cffa5274ed572974963)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:19.191615 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d84b0-3ed4-464a-a12b-d4625106ee8c admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:19.214167 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd7a26a-1eac-4d7d-9165-870e151436c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:19.214978 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd7a26a-1eac-4d7d-9165-870e151436c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:19.215359 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd7a26a-1eac-4d7d-9165-870e151436c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:19.217579 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd7a26a-1eac-4d7d-9165-870e151436c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:19.218482 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd7a26a-1eac-4d7d-9165-870e151436c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:19.221239 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd7a26a-1eac-4d7d-9165-870e151436c4 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:19.256442 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 870/1732] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:19.257766 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e799e4a-443c-43cb-9291-b2f9c62ddbc8 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:19.261986 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e799e4a-443c-43cb-9291-b2f9c62ddbc8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:19.262936 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e799e4a-443c-43cb-9291-b2f9c62ddbc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:19.315156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-7e799e4a-443c-43cb-9291-b2f9c62ddbc8 None None] MFA Rules not processed for user `a95746e7a84541199b43138b1a348386`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:19.317015 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7e799e4a-443c-43cb-9291-b2f9c62ddbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:19.334789 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-4d1d84b0-3ed4-464a-a12b-d4625106ee8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7231b818f44b4cffa5274ed572974963', 'request_id': 'req-4d1d84b0-3ed4-464a-a12b-d4625106ee8c'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:19.345935 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-4d1d84b0-3ed4-464a-a12b-d4625106ee8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7231b818f44b4cffa5274ed572974963', 'request_id': 'req-4d1d84b0-3ed4-464a-a12b-d4625106ee8c'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:19.357146 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-4d1d84b0-3ed4-464a-a12b-d4625106ee8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7231b818f44b4cffa5274ed572974963', 'request_id': 'req-4d1d84b0-3ed4-464a-a12b-d4625106ee8c'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:19.378889 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-4d1d84b0-3ed4-464a-a12b-d4625106ee8c admin admin] The token cache is being invalidate because project 7231b818f44b4cffa5274ed572974963 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:28:19.379552 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-4d1d84b0-3ed4-464a-a12b-d4625106ee8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:19.380250 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-4d1d84b0-3ed4-464a-a12b-d4625106ee8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:19.386977 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 863/1733] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:28:19 2022] DELETE /identity/v3/projects/7231b818f44b4cffa5274ed572974963 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:19.391266 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50592854-ac14-4d5f-b7c9-80cd46885993 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:19.391266 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50592854-ac14-4d5f-b7c9-80cd46885993 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:19.391266 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50592854-ac14-4d5f-b7c9-80cd46885993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:19.400894 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 871/1734] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:19 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:19.407341 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e799e4a-443c-43cb-9291-b2f9c62ddbc8 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:19.409331 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7e799e4a-443c-43cb-9291-b2f9c62ddbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:19.414705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-50592854-ac14-4d5f-b7c9-80cd46885993 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:19.424821 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-50592854-ac14-4d5f-b7c9-80cd46885993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:19.527395 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 864/1735] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:19.529005 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50592854-ac14-4d5f-b7c9-80cd46885993 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:19.532290 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-50592854-ac14-4d5f-b7c9-80cd46885993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:19.550954 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e799e4a-443c-43cb-9291-b2f9c62ddbc8 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:19.552414 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50592854-ac14-4d5f-b7c9-80cd46885993 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:19.560251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9bcd3d-bb16-4c23-9c72-59bf3e6cbedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:19.561091 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9bcd3d-bb16-4c23-9c72-59bf3e6cbedd admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:19.561948 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9bcd3d-bb16-4c23-9c72-59bf3e6cbedd admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:19.562229 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9bcd3d-bb16-4c23-9c72-59bf3e6cbedd admin admin] PATH_INFO: `/v3/projects/37588baaad34495ca5e4adc1e2cbaecc/users/34441ca279ee43919616f3d5eb199344/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:19.564051 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c8363-cabc-409f-b00a-151fd9c43761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:19.564947 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c8363-cabc-409f-b00a-151fd9c43761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:19.565474 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c8363-cabc-409f-b00a-151fd9c43761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:19.566071 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c8363-cabc-409f-b00a-151fd9c43761 admin admin] PATH_INFO: `/v3/projects/52becdc909ff44c4bf026a2ff9ee165a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:19.567675 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c8363-cabc-409f-b00a-151fd9c43761 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52becdc909ff44c4bf026a2ff9ee165a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:19.571374 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c8363-cabc-409f-b00a-151fd9c43761 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:19.588204 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9bcd3d-bb16-4c23-9c72-59bf3e6cbedd admin admin] RBAC: Authorizing `identity:create_grant(project_id=37588baaad34495ca5e4adc1e2cbaecc, user_id=34441ca279ee43919616f3d5eb199344, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:19.591693 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9bcd3d-bb16-4c23-9c72-59bf3e6cbedd admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:19.614995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 865/1736] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:19 2022] PUT /identity/v3/projects/37588baaad34495ca5e4adc1e2cbaecc/users/34441ca279ee43919616f3d5eb199344/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:19.616552 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361863d-5d22-445f-aa57-0b1e148f611c None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:19.616863 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361863d-5d22-445f-aa57-0b1e148f611c None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:19.617305 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361863d-5d22-445f-aa57-0b1e148f611c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:19.651885 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-9361863d-5d22-445f-aa57-0b1e148f611c None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:19.653483 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-9361863d-5d22-445f-aa57-0b1e148f611c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:19.680366 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-fc5c8363-cabc-409f-b00a-151fd9c43761 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52becdc909ff44c4bf026a2ff9ee165a', 'request_id': 'req-fc5c8363-cabc-409f-b00a-151fd9c43761'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:19.690293 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-fc5c8363-cabc-409f-b00a-151fd9c43761 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52becdc909ff44c4bf026a2ff9ee165a', 'request_id': 'req-fc5c8363-cabc-409f-b00a-151fd9c43761'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:19.699905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-fc5c8363-cabc-409f-b00a-151fd9c43761 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52becdc909ff44c4bf026a2ff9ee165a', 'request_id': 'req-fc5c8363-cabc-409f-b00a-151fd9c43761'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:19.723058 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:19.732752 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-fc5c8363-cabc-409f-b00a-151fd9c43761 admin admin] The token cache is being invalidate because project 52becdc909ff44c4bf026a2ff9ee165a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:28:19.733049 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-fc5c8363-cabc-409f-b00a-151fd9c43761 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:19.735133 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-fc5c8363-cabc-409f-b00a-151fd9c43761 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:19.737587 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 872/1737] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:28:19 2022] DELETE /identity/v3/projects/52becdc909ff44c4bf026a2ff9ee165a => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:19.738843 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f59504-a6ef-4b21-9cd1-d64c408e8f68 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:19.739122 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f59504-a6ef-4b21-9cd1-d64c408e8f68 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:19.739908 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f59504-a6ef-4b21-9cd1-d64c408e8f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:19.757612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:19.809175 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-41f59504-a6ef-4b21-9cd1-d64c408e8f68 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:19.813433 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-41f59504-a6ef-4b21-9cd1-d64c408e8f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:19.832496 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 866/1738] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:19.833553 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9361863d-5d22-445f-aa57-0b1e148f611c None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:19.838538 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-9361863d-5d22-445f-aa57-0b1e148f611c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:19.879771 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 873/1739] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:19.884249 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f59504-a6ef-4b21-9cd1-d64c408e8f68 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:19.884249 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-41f59504-a6ef-4b21-9cd1-d64c408e8f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:19.923657 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f59504-a6ef-4b21-9cd1-d64c408e8f68 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:19.948590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227139a-a541-4a91-9c2c-23b7e3394b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:19.949256 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227139a-a541-4a91-9c2c-23b7e3394b9c admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:19.949553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227139a-a541-4a91-9c2c-23b7e3394b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:19.949835 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227139a-a541-4a91-9c2c-23b7e3394b9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:19.950488 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4227139a-a541-4a91-9c2c-23b7e3394b9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:19.957318 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4227139a-a541-4a91-9c2c-23b7e3394b9c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:20.003370 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 874/1740] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:20.003507 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9361863d-5d22-445f-aa57-0b1e148f611c None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:20.009805 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da316fee-ebb5-4a71-9ecc-b798739fee1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:20.010432 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da316fee-ebb5-4a71-9ecc-b798739fee1d service nova] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:20.010707 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da316fee-ebb5-4a71-9ecc-b798739fee1d service nova] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:20.010976 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da316fee-ebb5-4a71-9ecc-b798739fee1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:20.012891 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-da316fee-ebb5-4a71-9ecc-b798739fee1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:20.017171 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c835e4-e289-4a81-8f01-44908bd44699 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:20.017633 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c835e4-e289-4a81-8f01-44908bd44699 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:20.017833 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c835e4-e289-4a81-8f01-44908bd44699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:20.100235 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a9c835e4-e289-4a81-8f01-44908bd44699 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:20.101520 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a9c835e4-e289-4a81-8f01-44908bd44699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:20.133587 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 875/1741] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:20.140161 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62818e2f-d613-435b-9cf1-8313bbd62bdd None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:20.140453 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62818e2f-d613-435b-9cf1-8313bbd62bdd None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:20.140576 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da316fee-ebb5-4a71-9ecc-b798739fee1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:20.140747 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62818e2f-d613-435b-9cf1-8313bbd62bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:20.144319 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da316fee-ebb5-4a71-9ecc-b798739fee1d service nova] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:20.170528 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 867/1742] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:19 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:20.171744 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da316fee-ebb5-4a71-9ecc-b798739fee1d service nova] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:20.199715 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da316fee-ebb5-4a71-9ecc-b798739fee1d service nova] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:20.207876 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-62818e2f-d613-435b-9cf1-8313bbd62bdd None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:20.210605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-62818e2f-d613-435b-9cf1-8313bbd62bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:20.213464 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfcda38-cf8e-479a-ae13-e341246d4147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:20.214102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfcda38-cf8e-479a-ae13-e341246d4147 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:20.214574 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfcda38-cf8e-479a-ae13-e341246d4147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:20.214973 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfcda38-cf8e-479a-ae13-e341246d4147 admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:20.216054 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfcda38-cf8e-479a-ae13-e341246d4147 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:28:20.216818 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfcda38-cf8e-479a-ae13-e341246d4147 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:20.222102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfcda38-cf8e-479a-ae13-e341246d4147 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:20.228828 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 876/1743] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:20.229810 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62818e2f-d613-435b-9cf1-8313bbd62bdd None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:20.253945 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62818e2f-d613-435b-9cf1-8313bbd62bdd None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:20.259270 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 868/1744] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:28:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:28:20.265979 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649c599-84c2-47fb-8ec1-21eb44ff57ca None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:20.266256 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649c599-84c2-47fb-8ec1-21eb44ff57ca None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:20.266525 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649c599-84c2-47fb-8ec1-21eb44ff57ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:20.281708 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbea045-a178-4f6b-ae6f-12042617ce67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:20.282407 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbea045-a178-4f6b-ae6f-12042617ce67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:20.282691 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbea045-a178-4f6b-ae6f-12042617ce67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:20.282969 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbea045-a178-4f6b-ae6f-12042617ce67 admin admin] PATH_INFO: `/v3/projects/37588baaad34495ca5e4adc1e2cbaecc/users/34441ca279ee43919616f3d5eb199344/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:20.285995 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbea045-a178-4f6b-ae6f-12042617ce67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37588baaad34495ca5e4adc1e2cbaecc, user_id=34441ca279ee43919616f3d5eb199344, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:20.290562 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbea045-a178-4f6b-ae6f-12042617ce67 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:20.317992 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-f649c599-84c2-47fb-8ec1-21eb44ff57ca None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:20.337911 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-f649c599-84c2-47fb-8ec1-21eb44ff57ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:20.359267 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 877/1745] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:20 2022] PUT /identity/v3/projects/37588baaad34495ca5e4adc1e2cbaecc/users/34441ca279ee43919616f3d5eb199344/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:20.374192 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbea045-a178-4f6b-ae6f-12042617ce67 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:20.471260 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:20.516875 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 869/1746] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:20.525434 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f649c599-84c2-47fb-8ec1-21eb44ff57ca None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:20.562870 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f649c599-84c2-47fb-8ec1-21eb44ff57ca None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:20.579369 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3438461c-4dee-4735-b19e-636153a0c5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:20.580155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438461c-4dee-4735-b19e-636153a0c5b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:20.580459 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438461c-4dee-4735-b19e-636153a0c5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:20.580776 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438461c-4dee-4735-b19e-636153a0c5b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:20.581421 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3438461c-4dee-4735-b19e-636153a0c5b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:20.583381 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3438461c-4dee-4735-b19e-636153a0c5b0 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:20.614309 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbea045-a178-4f6b-ae6f-12042617ce67 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:20.641006 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e064f99-7867-45d0-b53f-d955a430f683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:20.642048 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e064f99-7867-45d0-b53f-d955a430f683 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:20.642563 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e064f99-7867-45d0-b53f-d955a430f683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:20.643217 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e064f99-7867-45d0-b53f-d955a430f683 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:20.644104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e064f99-7867-45d0-b53f-d955a430f683 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:28:20.644615 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e064f99-7867-45d0-b53f-d955a430f683 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:20.651204 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e064f99-7867-45d0-b53f-d955a430f683 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:20.654312 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 870/1747] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:28:20 2022] POST /identity/v3/projects => generated 361 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:20.665532 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cf238-01d0-4c7f-bdec-00ee4891c720 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:20.665919 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cf238-01d0-4c7f-bdec-00ee4891c720 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:20.666270 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cf238-01d0-4c7f-bdec-00ee4891c720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:20.678082 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 878/1748] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:28:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:28:20.679515 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8c897-4e84-47da-ad65-96e3aff38c1b None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:20.682265 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8c897-4e84-47da-ad65-96e3aff38c1b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:20.684244 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8c897-4e84-47da-ad65-96e3aff38c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:20.753685 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-ef7cf238-01d0-4c7f-bdec-00ee4891c720 None None] MFA Rules not processed for user `34441ca279ee43919616f3d5eb199344`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:20.756211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-ef7cf238-01d0-4c7f-bdec-00ee4891c720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:20.879894 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-b8c8c897-4e84-47da-ad65-96e3aff38c1b None None] MFA Rules not processed for user `cedaf156c5584d62beedde598c94f322`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:20.881165 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b8c8c897-4e84-47da-ad65-96e3aff38c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:20.934108 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 871/1749] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:20 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:20.950748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7cf238-01d0-4c7f-bdec-00ee4891c720 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:21.071347 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 879/1750] 10.222.0.26 () {58 vars in 1009 bytes} [Fri Sep 9 20:28:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:21.071992 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8c897-4e84-47da-ad65-96e3aff38c1b None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:21.119764 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7cf238-01d0-4c7f-bdec-00ee4891c720 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:21.128060 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:21.130824 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d049481-9664-4ade-a9b2-e4c2b415ca8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:21.131594 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d049481-9664-4ade-a9b2-e4c2b415ca8a admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:21.131768 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d049481-9664-4ade-a9b2-e4c2b415ca8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:21.132095 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d049481-9664-4ade-a9b2-e4c2b415ca8a admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:21.132768 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d049481-9664-4ade-a9b2-e4c2b415ca8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:28:21.133089 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d049481-9664-4ade-a9b2-e4c2b415ca8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:21.136946 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d049481-9664-4ade-a9b2-e4c2b415ca8a admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:21.199220 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 872/1751] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:28:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:28:21.200340 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d049481-9664-4ade-a9b2-e4c2b415ca8a admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:21.232413 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8c897-4e84-47da-ad65-96e3aff38c1b None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:21.236985 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d049481-9664-4ade-a9b2-e4c2b415ca8a admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:21.242115 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e21fd-b641-4650-a46c-a55854368043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:21.243669 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca92ad-7ac6-4c5a-b771-21734284631b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:21.244261 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e21fd-b641-4650-a46c-a55854368043 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:21.244374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca92ad-7ac6-4c5a-b771-21734284631b admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:21.244609 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca92ad-7ac6-4c5a-b771-21734284631b admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:21.244935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e21fd-b641-4650-a46c-a55854368043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:21.245127 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca92ad-7ac6-4c5a-b771-21734284631b admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:21.245786 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ca92ad-7ac6-4c5a-b771-21734284631b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:21.246298 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e21fd-b641-4650-a46c-a55854368043 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:21.249219 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ca92ad-7ac6-4c5a-b771-21734284631b admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:21.251100 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e21fd-b641-4650-a46c-a55854368043 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:21.254941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e21fd-b641-4650-a46c-a55854368043 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:21.303424 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 873/1752] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:28:21 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:21.304773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca92ad-7ac6-4c5a-b771-21734284631b admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:21.306855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-13ca92ad-7ac6-4c5a-b771-21734284631b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:21.315259 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 880/1753] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:28:21 2022] POST /identity/v3/users => generated 281 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:21.316329 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e21fd-b641-4650-a46c-a55854368043 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:21.328592 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e21fd-b641-4650-a46c-a55854368043 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:21.396145 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:21.463783 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca92ad-7ac6-4c5a-b771-21734284631b admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:21.474542 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dca689-d177-44f2-ad91-2528fa757257 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:21.475245 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dca689-d177-44f2-ad91-2528fa757257 service cinder] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:21.475632 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dca689-d177-44f2-ad91-2528fa757257 service cinder] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:21.476013 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dca689-d177-44f2-ad91-2528fa757257 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:21.489743 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe11446-e5a7-4012-888c-854cfa3781e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:21.490939 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe11446-e5a7-4012-888c-854cfa3781e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:21.491629 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe11446-e5a7-4012-888c-854cfa3781e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:21.494807 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe11446-e5a7-4012-888c-854cfa3781e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:21.495646 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe11446-e5a7-4012-888c-854cfa3781e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:21.497791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe11446-e5a7-4012-888c-854cfa3781e1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:21.565376 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 881/1754] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:28:21 2022] POST /identity/v3/projects => generated 373 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:21.566806 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6eb61-18ee-4722-a7ea-ad8c93baedff None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:21.567092 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6eb61-18ee-4722-a7ea-ad8c93baedff None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:21.567365 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6eb61-18ee-4722-a7ea-ad8c93baedff None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:21.571067 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 882/1755] 10.222.0.26 () {54 vars in 912 bytes} [Fri Sep 9 20:28:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:28:21.573525 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6eb61-18ee-4722-a7ea-ad8c93baedff None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:21.654495 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dca689-d177-44f2-ad91-2528fa757257 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:21.660764 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dca689-d177-44f2-ad91-2528fa757257 service cinder] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:21.675236 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 874/1756] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:21 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:21.681500 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dca689-d177-44f2-ad91-2528fa757257 service cinder] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:21.704626 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dca689-d177-44f2-ad91-2528fa757257 service cinder] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:21.714265 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a1187-e345-4f0c-8ca4-7ac6fe7a13bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:21.715270 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a1187-e345-4f0c-8ca4-7ac6fe7a13bc admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:21.716522 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a1187-e345-4f0c-8ca4-7ac6fe7a13bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:21.717059 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a1187-e345-4f0c-8ca4-7ac6fe7a13bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:21.717682 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6eb61-18ee-4722-a7ea-ad8c93baedff None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:21.719063 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a1187-e345-4f0c-8ca4-7ac6fe7a13bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:21.724061 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a1187-e345-4f0c-8ca4-7ac6fe7a13bc admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:21.749436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc042a1-a581-4c1c-b214-7de0a3ca9ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:21.749571 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 875/1757] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:21.749670 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc042a1-a581-4c1c-b214-7de0a3ca9ea1 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:21.749670 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc042a1-a581-4c1c-b214-7de0a3ca9ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:21.749670 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc042a1-a581-4c1c-b214-7de0a3ca9ea1 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:21.749914 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a1187-e345-4f0c-8ca4-7ac6fe7a13bc admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:21.749998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc042a1-a581-4c1c-b214-7de0a3ca9ea1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:21.749998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc042a1-a581-4c1c-b214-7de0a3ca9ea1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:21.750160 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-2e1a1187-e345-4f0c-8ca4-7ac6fe7a13bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:21.823939 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 883/1758] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:28:21 2022] POST /identity/v3/users => generated 290 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:21.824824 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc042a1-a581-4c1c-b214-7de0a3ca9ea1 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:21.867790 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc042a1-a581-4c1c-b214-7de0a3ca9ea1 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:21.884135 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e534f-211b-4ee4-b15e-a10366f375eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:21.885786 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e534f-211b-4ee4-b15e-a10366f375eb admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:21.886101 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e534f-211b-4ee4-b15e-a10366f375eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:21.886380 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e534f-211b-4ee4-b15e-a10366f375eb admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:21.887021 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e534f-211b-4ee4-b15e-a10366f375eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:21.893950 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e534f-211b-4ee4-b15e-a10366f375eb admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:21.942136 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 884/1759] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:28:21 2022] POST /identity/v3/users => generated 287 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:21.943806 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78cd96-6e9a-4cab-ab4c-ee2e397ecb94 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:21.944287 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78cd96-6e9a-4cab-ab4c-ee2e397ecb94 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:21.949576 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78cd96-6e9a-4cab-ab4c-ee2e397ecb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:21.989742 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a1187-e345-4f0c-8ca4-7ac6fe7a13bc admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:21.990763 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-3b78cd96-6e9a-4cab-ab4c-ee2e397ecb94 None None] MFA Rules not processed for user `ee5c6e51f2694bf5ae7ea097cf33fb3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:21.993155 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3b78cd96-6e9a-4cab-ab4c-ee2e397ecb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:22.009834 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:22.019032 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23277779-131e-4f9c-90df-9b5427976cca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb62d06f7e94e9295c54504ddabb1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:22.019673 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23277779-131e-4f9c-90df-9b5427976cca service swift] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:22.019956 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23277779-131e-4f9c-90df-9b5427976cca service swift] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:22.020230 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23277779-131e-4f9c-90df-9b5427976cca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:22.098601 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:22.119996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 885/1760] 10.222.0.26 () {58 vars in 1009 bytes} [Fri Sep 9 20:28:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:22.125529 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23277779-131e-4f9c-90df-9b5427976cca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:22.126719 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78cd96-6e9a-4cab-ab4c-ee2e397ecb94 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:22.132010 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23277779-131e-4f9c-90df-9b5427976cca service swift] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:22.149340 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 876/1761] 10.222.0.26 () {60 vars in 1384 bytes} [Fri Sep 9 20:28:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 20:28:22.152605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78cd96-6e9a-4cab-ab4c-ee2e397ecb94 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:22.153725 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23277779-131e-4f9c-90df-9b5427976cca service swift] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:22.165177 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d93e09-5653-4d67-b9eb-8214c371245e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:22.165847 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d93e09-5653-4d67-b9eb-8214c371245e admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:22.166130 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d93e09-5653-4d67-b9eb-8214c371245e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:22.166417 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d93e09-5653-4d67-b9eb-8214c371245e admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:22.167628 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d93e09-5653-4d67-b9eb-8214c371245e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:22.168752 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d93e09-5653-4d67-b9eb-8214c371245e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:22.203171 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23277779-131e-4f9c-90df-9b5427976cca service swift] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:22.207575 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 886/1762] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:22.208503 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d93e09-5653-4d67-b9eb-8214c371245e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:22.215730 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e508df-fc04-45cc-b345-15cba914629a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:22.216800 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e508df-fc04-45cc-b345-15cba914629a admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:22.217177 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e508df-fc04-45cc-b345-15cba914629a admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:22.232464 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e508df-fc04-45cc-b345-15cba914629a admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:22.235153 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e508df-fc04-45cc-b345-15cba914629a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:22.244297 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e508df-fc04-45cc-b345-15cba914629a admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:22.271525 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 877/1763] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:22.275356 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e508df-fc04-45cc-b345-15cba914629a admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:22.302483 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e508df-fc04-45cc-b345-15cba914629a admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:22.317537 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b714904-cd99-45c5-af45-713dca18ba72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:22.317537 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b714904-cd99-45c5-af45-713dca18ba72 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:22.320174 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b714904-cd99-45c5-af45-713dca18ba72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:22.320174 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b714904-cd99-45c5-af45-713dca18ba72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:22.322509 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b714904-cd99-45c5-af45-713dca18ba72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:22.326483 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b714904-cd99-45c5-af45-713dca18ba72 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:22.358375 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 878/1764] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:22.358375 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b714904-cd99-45c5-af45-713dca18ba72 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:22.358375 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-5b714904-cd99-45c5-af45-713dca18ba72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:22.486737 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b714904-cd99-45c5-af45-713dca18ba72 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:22.492295 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d93e09-5653-4d67-b9eb-8214c371245e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:22.498210 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a9f7fc-4173-4fff-a2a6-107c39dd66d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:22.499550 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a9f7fc-4173-4fff-a2a6-107c39dd66d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:22.499718 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a9f7fc-4173-4fff-a2a6-107c39dd66d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:22.500403 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a9f7fc-4173-4fff-a2a6-107c39dd66d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:22.516600 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a9f7fc-4173-4fff-a2a6-107c39dd66d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:22.517805 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed77809-d33b-4508-a072-67203bcbac95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:22.520636 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed77809-d33b-4508-a072-67203bcbac95 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:22.520852 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a9f7fc-4173-4fff-a2a6-107c39dd66d6 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:22.520970 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed77809-d33b-4508-a072-67203bcbac95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:22.521206 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed77809-d33b-4508-a072-67203bcbac95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:22.539918 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 879/1765] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:22.541018 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4ed77809-d33b-4508-a072-67203bcbac95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:22.543202 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a9f7fc-4173-4fff-a2a6-107c39dd66d6 service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:22.572980 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a9f7fc-4173-4fff-a2a6-107c39dd66d6 service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:22.593744 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c5c982-f0e1-4a89-87c9-0470a8583b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:22.598142 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5c982-f0e1-4a89-87c9-0470a8583b80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:22.602876 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5c982-f0e1-4a89-87c9-0470a8583b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:22.603376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5c982-f0e1-4a89-87c9-0470a8583b80 admin admin] PATH_INFO: `/v3/projects/8f5e1579b6d3470eaf120e05c9a466dc/users/0f9e6f3182c8442895558414b5933ee5/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:22.688277 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c5c982-f0e1-4a89-87c9-0470a8583b80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f5e1579b6d3470eaf120e05c9a466dc, user_id=0f9e6f3182c8442895558414b5933ee5, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:22.701520 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c5c982-f0e1-4a89-87c9-0470a8583b80 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:22.777070 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 880/1766] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:22 2022] PUT /identity/v3/projects/8f5e1579b6d3470eaf120e05c9a466dc/users/0f9e6f3182c8442895558414b5933ee5/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:22.780605 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c5c982-f0e1-4a89-87c9-0470a8583b80 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:22.837413 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed77809-d33b-4508-a072-67203bcbac95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:22.838889 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed77809-d33b-4508-a072-67203bcbac95 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:22.885673 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 887/1767] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 677 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:22.886631 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed77809-d33b-4508-a072-67203bcbac95 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:22.920460 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed77809-d33b-4508-a072-67203bcbac95 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:23.064299 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c5c982-f0e1-4a89-87c9-0470a8583b80 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:23.086446 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f6f6ad-720b-4b9c-b0b3-4fbcdbafd918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:23.091547 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6f6ad-720b-4b9c-b0b3-4fbcdbafd918 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:23.095168 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6f6ad-720b-4b9c-b0b3-4fbcdbafd918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:23.096261 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6f6ad-720b-4b9c-b0b3-4fbcdbafd918 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:23.100662 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f6f6ad-720b-4b9c-b0b3-4fbcdbafd918 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:23.106304 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f6f6ad-720b-4b9c-b0b3-4fbcdbafd918 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:23.170244 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053b44e-a5bf-44c6-b73c-ca7f09573aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:23.171623 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053b44e-a5bf-44c6-b73c-ca7f09573aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:23.172080 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053b44e-a5bf-44c6-b73c-ca7f09573aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:23.172574 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053b44e-a5bf-44c6-b73c-ca7f09573aa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:23.173515 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4053b44e-a5bf-44c6-b73c-ca7f09573aa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:23.175691 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4053b44e-a5bf-44c6-b73c-ca7f09573aa1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:23.182973 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 881/1768] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:23.191328 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:23.200494 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043bbce-29fb-4a15-b662-1474b0972a69 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:23.200848 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043bbce-29fb-4a15-b662-1474b0972a69 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:23.201135 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043bbce-29fb-4a15-b662-1474b0972a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:23.205740 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 888/1769] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:23.208907 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053b44e-a5bf-44c6-b73c-ca7f09573aa1 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:23.269268 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053b44e-a5bf-44c6-b73c-ca7f09573aa1 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:23.296963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa6401-0202-4097-9b93-c49f6b3c0956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:23.310027 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa6401-0202-4097-9b93-c49f6b3c0956 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:23.310375 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa6401-0202-4097-9b93-c49f6b3c0956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:23.310596 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa6401-0202-4097-9b93-c49f6b3c0956 admin admin] PATH_INFO: `/v3/projects/44c6b1ae2e8546f8bc57ebcfb259ed32/users/644451c216c74e818005e66122d79a00/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:23.354932 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-0043bbce-29fb-4a15-b662-1474b0972a69 None None] MFA Rules not processed for user `0f9e6f3182c8442895558414b5933ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:23.356461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-0043bbce-29fb-4a15-b662-1474b0972a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:23.404439 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa6401-0202-4097-9b93-c49f6b3c0956 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44c6b1ae2e8546f8bc57ebcfb259ed32, user_id=644451c216c74e818005e66122d79a00, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:23.420324 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa6401-0202-4097-9b93-c49f6b3c0956 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:23.486697 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 889/1770] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:23 2022] PUT /identity/v3/projects/44c6b1ae2e8546f8bc57ebcfb259ed32/users/644451c216c74e818005e66122d79a00/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:23.487960 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa6401-0202-4097-9b93-c49f6b3c0956 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:23.687756 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 882/1771] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 500 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:23.693889 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217e939-599f-447a-9332-e95eebacb501 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:23.694190 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217e939-599f-447a-9332-e95eebacb501 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:23.694429 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217e939-599f-447a-9332-e95eebacb501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:23.699558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa6401-0202-4097-9b93-c49f6b3c0956 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:23.714156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef5141-f8a1-4e9e-83d5-98eaf88f6a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:23.714905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef5141-f8a1-4e9e-83d5-98eaf88f6a56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:23.715128 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef5141-f8a1-4e9e-83d5-98eaf88f6a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:23.715407 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef5141-f8a1-4e9e-83d5-98eaf88f6a56 admin admin] PATH_INFO: `/v3/projects/3422e322ea884f7e943bc20378e4bbf7/users/371d5028558649569e6318e826e1e8e9/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:23.770032 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef5141-f8a1-4e9e-83d5-98eaf88f6a56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3422e322ea884f7e943bc20378e4bbf7, user_id=371d5028558649569e6318e826e1e8e9, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:23.773227 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef5141-f8a1-4e9e-83d5-98eaf88f6a56 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:23.785492 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:23.801590 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-a217e939-599f-447a-9332-e95eebacb501 None None] MFA Rules not processed for user `644451c216c74e818005e66122d79a00`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:23.802825 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a217e939-599f-447a-9332-e95eebacb501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:23.810337 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 890/1772] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:23 2022] PUT /identity/v3/projects/3422e322ea884f7e943bc20378e4bbf7/users/371d5028558649569e6318e826e1e8e9/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:23.837193 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef5141-f8a1-4e9e-83d5-98eaf88f6a56 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:23.955915 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 883/1773] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:23.962624 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515fe5f9-91bc-40a0-9fe0-f6b3cd2b7bfa None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:23.962910 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515fe5f9-91bc-40a0-9fe0-f6b3cd2b7bfa None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:23.963179 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515fe5f9-91bc-40a0-9fe0-f6b3cd2b7bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:24.066035 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-515fe5f9-91bc-40a0-9fe0-f6b3cd2b7bfa None None] MFA Rules not processed for user `371d5028558649569e6318e826e1e8e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:24.096860 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-515fe5f9-91bc-40a0-9fe0-f6b3cd2b7bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:24.120973 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef5141-f8a1-4e9e-83d5-98eaf88f6a56 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:24.153637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce02b3-b2f2-4e8d-b456-81667eea56be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:24.154340 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce02b3-b2f2-4e8d-b456-81667eea56be admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:24.154558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce02b3-b2f2-4e8d-b456-81667eea56be admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:24.154836 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce02b3-b2f2-4e8d-b456-81667eea56be admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:24.159311 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce02b3-b2f2-4e8d-b456-81667eea56be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:24.163789 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce02b3-b2f2-4e8d-b456-81667eea56be admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:24.246843 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 891/1774] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:28:23 2022] POST /identity/v3/projects => generated 363 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:24.248856 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce02b3-b2f2-4e8d-b456-81667eea56be admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:24.255852 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 884/1775] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:24.257536 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e557b7-857d-4b09-950f-9fbfdf0a4e1d None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:24.257663 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e557b7-857d-4b09-950f-9fbfdf0a4e1d None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:24.257857 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e557b7-857d-4b09-950f-9fbfdf0a4e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:24.312198 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-87e557b7-857d-4b09-950f-9fbfdf0a4e1d None None] MFA Rules not processed for user `ee5c6e51f2694bf5ae7ea097cf33fb3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:24.314222 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-87e557b7-857d-4b09-950f-9fbfdf0a4e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:24.399559 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:24.443108 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce02b3-b2f2-4e8d-b456-81667eea56be admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:24.458996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edccf6b-d2f3-4a29-855b-cb22d873cf90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:24.459641 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edccf6b-d2f3-4a29-855b-cb22d873cf90 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:24.459929 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edccf6b-d2f3-4a29-855b-cb22d873cf90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:24.460206 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edccf6b-d2f3-4a29-855b-cb22d873cf90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:24.460868 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edccf6b-d2f3-4a29-855b-cb22d873cf90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:24.478383 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 885/1776] 10.222.0.25 () {58 vars in 1015 bytes} [Fri Sep 9 20:28:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:24.479226 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e557b7-857d-4b09-950f-9fbfdf0a4e1d None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:24.481697 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edccf6b-d2f3-4a29-855b-cb22d873cf90 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:24.505070 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e557b7-857d-4b09-950f-9fbfdf0a4e1d None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:24.520727 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1fa266-a368-4654-96a7-e6c8f0cc0c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:24.525543 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1fa266-a368-4654-96a7-e6c8f0cc0c4d admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:24.525950 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1fa266-a368-4654-96a7-e6c8f0cc0c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:24.526233 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1fa266-a368-4654-96a7-e6c8f0cc0c4d admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:24.531323 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1fa266-a368-4654-96a7-e6c8f0cc0c4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:24.547085 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1fa266-a368-4654-96a7-e6c8f0cc0c4d admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:24.570197 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 892/1777] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:28:24 2022] POST /identity/v3/projects => generated 379 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:24.571361 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edccf6b-d2f3-4a29-855b-cb22d873cf90 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:24.603587 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 886/1778] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:28:24 2022] POST /identity/v3/users => generated 288 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:24.605556 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1fa266-a368-4654-96a7-e6c8f0cc0c4d admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:24.631268 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1fa266-a368-4654-96a7-e6c8f0cc0c4d admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:24.681647 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:24.704807 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edccf6b-d2f3-4a29-855b-cb22d873cf90 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:24.710842 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24bd7cb-8f37-48c4-ac99-22046886f43e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:24.711701 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24bd7cb-8f37-48c4-ac99-22046886f43e service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:24.712120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24bd7cb-8f37-48c4-ac99-22046886f43e service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:24.713232 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24bd7cb-8f37-48c4-ac99-22046886f43e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:24.861500 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24bd7cb-8f37-48c4-ac99-22046886f43e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:24.864309 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24bd7cb-8f37-48c4-ac99-22046886f43e service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:24.864896 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b90cb-aca2-4b76-a3b0-24d47877ad34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:24.867039 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b90cb-aca2-4b76-a3b0-24d47877ad34 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:24.867476 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b90cb-aca2-4b76-a3b0-24d47877ad34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:24.869894 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b90cb-aca2-4b76-a3b0-24d47877ad34 admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:24.870798 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b90cb-aca2-4b76-a3b0-24d47877ad34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:24.878591 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b90cb-aca2-4b76-a3b0-24d47877ad34 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:24.896474 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 893/1779] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:24.897888 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24bd7cb-8f37-48c4-ac99-22046886f43e service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:24.926997 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24bd7cb-8f37-48c4-ac99-22046886f43e service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:24.939143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 887/1780] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:28:24 2022] POST /identity/v3/users => generated 296 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:24.939997 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b90cb-aca2-4b76-a3b0-24d47877ad34 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:24.944663 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8de289-6075-4fe0-885a-abc0abaa6398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:24.947082 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8de289-6075-4fe0-885a-abc0abaa6398 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:24.947544 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8de289-6075-4fe0-885a-abc0abaa6398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:24.948421 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8de289-6075-4fe0-885a-abc0abaa6398 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:24.949697 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8de289-6075-4fe0-885a-abc0abaa6398 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:24.953373 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8de289-6075-4fe0-885a-abc0abaa6398 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:24.999175 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 894/1781] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:24 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:25.001807 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8de289-6075-4fe0-885a-abc0abaa6398 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:25.040006 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8de289-6075-4fe0-885a-abc0abaa6398 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:25.051256 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80718b93-35f1-4ee4-ab21-d70fbb09b7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:25.052158 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80718b93-35f1-4ee4-ab21-d70fbb09b7ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:25.052439 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80718b93-35f1-4ee4-ab21-d70fbb09b7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:25.052712 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80718b93-35f1-4ee4-ab21-d70fbb09b7ce admin admin] PATH_INFO: `/v3/users/371d5028558649569e6318e826e1e8e9` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:25.054884 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80718b93-35f1-4ee4-ab21-d70fbb09b7ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=371d5028558649569e6318e826e1e8e9)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:25.059489 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80718b93-35f1-4ee4-ab21-d70fbb09b7ce admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:25.116928 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b90cb-aca2-4b76-a3b0-24d47877ad34 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:25.122652 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5972be7-deeb-470c-a0a3-70943d109a69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:25.123406 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5972be7-deeb-470c-a0a3-70943d109a69 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:25.123820 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5972be7-deeb-470c-a0a3-70943d109a69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:25.124100 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5972be7-deeb-470c-a0a3-70943d109a69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:25.135099 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5972be7-deeb-470c-a0a3-70943d109a69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:25.137084 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5972be7-deeb-470c-a0a3-70943d109a69 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:25.159721 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-80718b93-35f1-4ee4-ab21-d70fbb09b7ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '371d5028558649569e6318e826e1e8e9'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:25.188397 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-80718b93-35f1-4ee4-ab21-d70fbb09b7ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '371d5028558649569e6318e826e1e8e9'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:25.189113 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 888/1782] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:25.192166 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5972be7-deeb-470c-a0a3-70943d109a69 service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:25.210851 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:25.223725 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-80718b93-35f1-4ee4-ab21-d70fbb09b7ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '371d5028558649569e6318e826e1e8e9'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:25.224940 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5972be7-deeb-470c-a0a3-70943d109a69 service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:25.260531 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce05238-b264-4fe7-8207-98b3bfa80abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:25.263474 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce05238-b264-4fe7-8207-98b3bfa80abe admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:25.265748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce05238-b264-4fe7-8207-98b3bfa80abe admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:25.267556 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce05238-b264-4fe7-8207-98b3bfa80abe admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:25.268552 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce05238-b264-4fe7-8207-98b3bfa80abe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:25.274373 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-80718b93-35f1-4ee4-ab21-d70fbb09b7ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '371d5028558649569e6318e826e1e8e9'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:25.275307 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-80718b93-35f1-4ee4-ab21-d70fbb09b7ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '371d5028558649569e6318e826e1e8e9'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:25.276788 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce05238-b264-4fe7-8207-98b3bfa80abe admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:25.307558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 895/1783] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:28:24 2022] DELETE /identity/v3/users/371d5028558649569e6318e826e1e8e9 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:25.312000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80718b93-35f1-4ee4-ab21-d70fbb09b7ce admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:25.315904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-80718b93-35f1-4ee4-ab21-d70fbb09b7ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:25.320499 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 889/1784] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:25.329623 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3187a98-c201-44e3-98d7-5d9e9498204d None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:25.332328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3187a98-c201-44e3-98d7-5d9e9498204d None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:25.332793 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3187a98-c201-44e3-98d7-5d9e9498204d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:25.420401 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-d3187a98-c201-44e3-98d7-5d9e9498204d None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:25.431315 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-d3187a98-c201-44e3-98d7-5d9e9498204d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:25.719664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80718b93-35f1-4ee4-ab21-d70fbb09b7ce admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:25.762887 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab21fdc-b1e4-4eb3-9969-cd88ba5e243c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:25.766594 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab21fdc-b1e4-4eb3-9969-cd88ba5e243c admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:25.766882 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab21fdc-b1e4-4eb3-9969-cd88ba5e243c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:25.771422 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab21fdc-b1e4-4eb3-9969-cd88ba5e243c admin admin] PATH_INFO: `/v3/projects/e85c36d05d3b4bee8645d3d65bf46b2f/users/2874613639ab4388ac76c9b29cccb60f/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:25.846338 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 890/1785] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 522 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:25.846510 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3187a98-c201-44e3-98d7-5d9e9498204d None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:25.862775 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab21fdc-b1e4-4eb3-9969-cd88ba5e243c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e85c36d05d3b4bee8645d3d65bf46b2f, user_id=2874613639ab4388ac76c9b29cccb60f, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:25.865833 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-d3187a98-c201-44e3-98d7-5d9e9498204d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:25.874226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab21fdc-b1e4-4eb3-9969-cd88ba5e243c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:25.937031 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3187a98-c201-44e3-98d7-5d9e9498204d None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:25.939988 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 896/1786] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:25 2022] PUT /identity/v3/projects/e85c36d05d3b4bee8645d3d65bf46b2f/users/2874613639ab4388ac76c9b29cccb60f/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:25.943509 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669360a9-24aa-4aef-8b7f-f794454e934c None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:25.943843 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669360a9-24aa-4aef-8b7f-f794454e934c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:25.944124 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669360a9-24aa-4aef-8b7f-f794454e934c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:25.991551 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-669360a9-24aa-4aef-8b7f-f794454e934c None None] MFA Rules not processed for user `a95746e7a84541199b43138b1a348386`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:26.001372 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-669360a9-24aa-4aef-8b7f-f794454e934c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:26.053625 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:26.087068 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec88b3e-5565-4398-8301-c9a27f2df244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:26.087593 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec88b3e-5565-4398-8301-c9a27f2df244 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:26.087877 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec88b3e-5565-4398-8301-c9a27f2df244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:26.088158 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec88b3e-5565-4398-8301-c9a27f2df244 admin admin] PATH_INFO: `/v3/projects/1f93ddca81f54b3d8d48de887071e624/users/770fff565e854632af1cdfb281fc1575/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:26.118372 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec88b3e-5565-4398-8301-c9a27f2df244 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f93ddca81f54b3d8d48de887071e624, user_id=770fff565e854632af1cdfb281fc1575, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:26.121124 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec88b3e-5565-4398-8301-c9a27f2df244 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:26.160641 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 897/1787] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:25 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:26.166545 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 891/1788] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:25 2022] PUT /identity/v3/projects/1f93ddca81f54b3d8d48de887071e624/users/770fff565e854632af1cdfb281fc1575/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:26.167462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec88b3e-5565-4398-8301-c9a27f2df244 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:26.174160 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-eec88b3e-5565-4398-8301-c9a27f2df244 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:26.174365 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4a9b2-af6d-48fb-9736-da62aedfb695 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:26.174365 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4a9b2-af6d-48fb-9736-da62aedfb695 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:26.174365 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4a9b2-af6d-48fb-9736-da62aedfb695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:26.201862 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-8ea4a9b2-af6d-48fb-9736-da62aedfb695 None None] MFA Rules not processed for user `db1f365ebf794941b3999adb37c48100`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:26.203171 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8ea4a9b2-af6d-48fb-9736-da62aedfb695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:26.296704 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 898/1789] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:26.301218 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4a9b2-af6d-48fb-9736-da62aedfb695 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:26.322152 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec88b3e-5565-4398-8301-c9a27f2df244 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:26.328007 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b384b37-fbec-4b7f-aec1-00486192cc80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:26.328875 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b384b37-fbec-4b7f-aec1-00486192cc80 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:26.329279 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b384b37-fbec-4b7f-aec1-00486192cc80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:26.329784 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b384b37-fbec-4b7f-aec1-00486192cc80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:26.403267 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:26.473988 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b384b37-fbec-4b7f-aec1-00486192cc80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:26.476765 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b384b37-fbec-4b7f-aec1-00486192cc80 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:26.496547 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 892/1790] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:26.498256 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b384b37-fbec-4b7f-aec1-00486192cc80 service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:26.539546 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b384b37-fbec-4b7f-aec1-00486192cc80 service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:26.550093 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4a9b2-af6d-48fb-9736-da62aedfb695 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:26.556287 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9075a-5a20-4e70-8503-087c9bdab013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:26.557332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9075a-5a20-4e70-8503-087c9bdab013 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:26.557769 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9075a-5a20-4e70-8503-087c9bdab013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:26.558060 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9075a-5a20-4e70-8503-087c9bdab013 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:26.558863 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f9075a-5a20-4e70-8503-087c9bdab013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:26.560436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f9075a-5a20-4e70-8503-087c9bdab013 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:26.563440 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70109566-b06f-4e87-987e-cbb9821592c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:26.564200 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70109566-b06f-4e87-987e-cbb9821592c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:26.564374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70109566-b06f-4e87-987e-cbb9821592c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:26.564655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70109566-b06f-4e87-987e-cbb9821592c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:26.565481 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70109566-b06f-4e87-987e-cbb9821592c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:26.566997 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70109566-b06f-4e87-987e-cbb9821592c1 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:26.576331 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 899/1791] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:26.580033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9075a-5a20-4e70-8503-087c9bdab013 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:26.597001 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9075a-5a20-4e70-8503-087c9bdab013 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:26.604171 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 893/1792] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:26.625204 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b9fb0b-51d7-43d7-9c97-25a605075ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:26.626935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9fb0b-51d7-43d7-9c97-25a605075ff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:26.627327 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9fb0b-51d7-43d7-9c97-25a605075ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:26.627731 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9fb0b-51d7-43d7-9c97-25a605075ff4 admin admin] PATH_INFO: `/v3/projects/e85c36d05d3b4bee8645d3d65bf46b2f/users/2874613639ab4388ac76c9b29cccb60f/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:26.628462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70109566-b06f-4e87-987e-cbb9821592c1 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:26.632393 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b9fb0b-51d7-43d7-9c97-25a605075ff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e85c36d05d3b4bee8645d3d65bf46b2f, user_id=2874613639ab4388ac76c9b29cccb60f, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:26.637547 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b9fb0b-51d7-43d7-9c97-25a605075ff4 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:26.660126 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70109566-b06f-4e87-987e-cbb9821592c1 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:26.660325 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 900/1793] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:26 2022] PUT /identity/v3/projects/e85c36d05d3b4bee8645d3d65bf46b2f/users/2874613639ab4388ac76c9b29cccb60f/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:28:26.672023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b86f2b-5351-4732-81ff-1ba938d5cd3e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:26.681997 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b86f2b-5351-4732-81ff-1ba938d5cd3e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:26.682376 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b86f2b-5351-4732-81ff-1ba938d5cd3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:26.791564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e2b86f2b-5351-4732-81ff-1ba938d5cd3e None None] MFA Rules not processed for user `2874613639ab4388ac76c9b29cccb60f`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:26.792695 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e2b86f2b-5351-4732-81ff-1ba938d5cd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:26.910699 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9aebf6-f025-4c4c-ab1c-9c1b833517ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:26.910699 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9aebf6-f025-4c4c-ab1c-9c1b833517ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:26.911365 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9aebf6-f025-4c4c-ab1c-9c1b833517ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:26.911365 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9aebf6-f025-4c4c-ab1c-9c1b833517ee admin admin] PATH_INFO: `/v3/projects/1f93ddca81f54b3d8d48de887071e624/users/770fff565e854632af1cdfb281fc1575/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:26.916527 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9aebf6-f025-4c4c-ab1c-9c1b833517ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f93ddca81f54b3d8d48de887071e624, user_id=770fff565e854632af1cdfb281fc1575, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:26.919249 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9aebf6-f025-4c4c-ab1c-9c1b833517ee admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:26.976030 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:26.997609 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 894/1794] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:26 2022] PUT /identity/v3/projects/1f93ddca81f54b3d8d48de887071e624/users/770fff565e854632af1cdfb281fc1575/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:27.010376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f0cb5-ed37-4cee-8f82-e2dcb407f20e None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:27.012883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f0cb5-ed37-4cee-8f82-e2dcb407f20e None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:27.013907 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f0cb5-ed37-4cee-8f82-e2dcb407f20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:27.098215 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 901/1795] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:26 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 428 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:27.106191 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b86f2b-5351-4732-81ff-1ba938d5cd3e None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:27.114154 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e2b86f2b-5351-4732-81ff-1ba938d5cd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:27.186959 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-d28f0cb5-ed37-4cee-8f82-e2dcb407f20e None None] MFA Rules not processed for user `770fff565e854632af1cdfb281fc1575`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:27.188212 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-d28f0cb5-ed37-4cee-8f82-e2dcb407f20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:27.308643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b86f2b-5351-4732-81ff-1ba938d5cd3e None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:27.431595 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:27.464464 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843cea9-7bfd-414a-b07e-f3d4f6bbe1a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:27.472700 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843cea9-7bfd-414a-b07e-f3d4f6bbe1a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:27.472700 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843cea9-7bfd-414a-b07e-f3d4f6bbe1a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:27.472700 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843cea9-7bfd-414a-b07e-f3d4f6bbe1a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:27.496820 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 895/1796] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:27 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 488 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:27.714185 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8843cea9-7bfd-414a-b07e-f3d4f6bbe1a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:27.716026 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8843cea9-7bfd-414a-b07e-f3d4f6bbe1a0 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:27.752630 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 902/1797] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:27 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 647 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:27.832137 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f0cb5-ed37-4cee-8f82-e2dcb407f20e None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:27.834328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-d28f0cb5-ed37-4cee-8f82-e2dcb407f20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:27.861695 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f0cb5-ed37-4cee-8f82-e2dcb407f20e None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:27.878068 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572d4b9-09c0-4e27-b5fe-5097f55b08f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:27.899677 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572d4b9-09c0-4e27-b5fe-5097f55b08f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:27.900680 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572d4b9-09c0-4e27-b5fe-5097f55b08f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:27.903494 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572d4b9-09c0-4e27-b5fe-5097f55b08f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:27.954868 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7572d4b9-09c0-4e27-b5fe-5097f55b08f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:27.962621 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7572d4b9-09c0-4e27-b5fe-5097f55b08f7 service cinder] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:28.013983 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 896/1798] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:27 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:28.015886 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843cea9-7bfd-414a-b07e-f3d4f6bbe1a0 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:28.048968 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843cea9-7bfd-414a-b07e-f3d4f6bbe1a0 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:28.070402 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b66646-b050-4655-b83c-82a593159f57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:28.071151 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b66646-b050-4655-b83c-82a593159f57 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:28.074255 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b66646-b050-4655-b83c-82a593159f57 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:28.074689 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b66646-b050-4655-b83c-82a593159f57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:28.079304 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-22b66646-b050-4655-b83c-82a593159f57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:28.307447 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b66646-b050-4655-b83c-82a593159f57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:28.309069 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b66646-b050-4655-b83c-82a593159f57 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:28.351272 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 903/1799] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:28.733577 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572d4b9-09c0-4e27-b5fe-5097f55b08f7 service cinder] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:28.745637 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572d4b9-09c0-4e27-b5fe-5097f55b08f7 service cinder] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:28.760392 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f09d0c-e850-4e51-a720-70a89679fb83 tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770fff565e854632af1cdfb281fc1575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f93ddca81f54b3d8d48de887071e624', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:28.761120 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f09d0c-e850-4e51-a720-70a89679fb83 tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:28.761810 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f09d0c-e850-4e51-a720-70a89679fb83 tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:28.761810 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f09d0c-e850-4e51-a720-70a89679fb83 tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:28.762785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f09d0c-e850-4e51-a720-70a89679fb83 tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:28:28.763103 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f09d0c-e850-4e51-a720-70a89679fb83 tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:28.765488 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f09d0c-e850-4e51-a720-70a89679fb83 tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:28.801513 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 897/1800] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:28:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:28:28.806040 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b66646-b050-4655-b83c-82a593159f57 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:28.825601 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b66646-b050-4655-b83c-82a593159f57 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:28.832933 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d89e2f-54a5-4e78-b160-805822b230ac tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770fff565e854632af1cdfb281fc1575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f93ddca81f54b3d8d48de887071e624', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:28.833568 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d89e2f-54a5-4e78-b160-805822b230ac tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:28.834014 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d89e2f-54a5-4e78-b160-805822b230ac tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:28.834530 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d89e2f-54a5-4e78-b160-805822b230ac tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:28.835379 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d89e2f-54a5-4e78-b160-805822b230ac tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:28.838216 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d89e2f-54a5-4e78-b160-805822b230ac tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:28.896131 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 904/1801] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:28:28 2022] POST /identity/v3/projects => generated 355 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:28:28.897051 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f09d0c-e850-4e51-a720-70a89679fb83 tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:28.915880 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f09d0c-e850-4e51-a720-70a89679fb83 tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:28.992254 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d89e2f-54a5-4e78-b160-805822b230ac tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:28.994193 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-35d89e2f-54a5-4e78-b160-805822b230ac tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:29.071038 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e18edd-d560-4a7d-ae91-1a248feba10f tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770fff565e854632af1cdfb281fc1575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f93ddca81f54b3d8d48de887071e624', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:29.071656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e18edd-d560-4a7d-ae91-1a248feba10f tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:29.072083 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e18edd-d560-4a7d-ae91-1a248feba10f tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:29.072376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e18edd-d560-4a7d-ae91-1a248feba10f tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:29.073041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e18edd-d560-4a7d-ae91-1a248feba10f tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:28:29.073458 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e18edd-d560-4a7d-ae91-1a248feba10f tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:29.084562 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e18edd-d560-4a7d-ae91-1a248feba10f tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:29.104948 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 898/1802] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:28:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:28:29.170446 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e18edd-d560-4a7d-ae91-1a248feba10f tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:29.188489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e18edd-d560-4a7d-ae91-1a248feba10f tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:29.205235 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d89e2f-54a5-4e78-b160-805822b230ac tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:29.206686 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5122db-425b-4bc9-85bb-e89c90622e57 tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770fff565e854632af1cdfb281fc1575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f93ddca81f54b3d8d48de887071e624', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:29.207928 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5122db-425b-4bc9-85bb-e89c90622e57 tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:29.208413 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5122db-425b-4bc9-85bb-e89c90622e57 tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:29.209050 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5122db-425b-4bc9-85bb-e89c90622e57 tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] PATH_INFO: `/v3/projects/957e49fead28485fa4fee73c14041710` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:29.211433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5122db-425b-4bc9-85bb-e89c90622e57 tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] RBAC: Authorizing `identity:delete_project(project_id=957e49fead28485fa4fee73c14041710)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:29.212874 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7aae42-c129-4be7-9967-e43904eccda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:29.213571 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7aae42-c129-4be7-9967-e43904eccda6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:29.213855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7aae42-c129-4be7-9967-e43904eccda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:29.214131 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7aae42-c129-4be7-9967-e43904eccda6 admin admin] PATH_INFO: `/v3/projects/3422e322ea884f7e943bc20378e4bbf7` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:29.216115 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5122db-425b-4bc9-85bb-e89c90622e57 tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:29.219545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7aae42-c129-4be7-9967-e43904eccda6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3422e322ea884f7e943bc20378e4bbf7)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:29.221778 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7aae42-c129-4be7-9967-e43904eccda6 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:29.252008 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:29.327939 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:29.331002 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-7a5122db-425b-4bc9-85bb-e89c90622e57 tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '957e49fead28485fa4fee73c14041710', 'request_id': 'req-7a5122db-425b-4bc9-85bb-e89c90622e57'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:29.338307 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bf7aae42-c129-4be7-9967-e43904eccda6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3422e322ea884f7e943bc20378e4bbf7', 'request_id': 'req-bf7aae42-c129-4be7-9967-e43904eccda6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:29.354240 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bf7aae42-c129-4be7-9967-e43904eccda6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3422e322ea884f7e943bc20378e4bbf7', 'request_id': 'req-bf7aae42-c129-4be7-9967-e43904eccda6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:29.359641 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-7a5122db-425b-4bc9-85bb-e89c90622e57 tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '957e49fead28485fa4fee73c14041710', 'request_id': 'req-7a5122db-425b-4bc9-85bb-e89c90622e57'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:29.368849 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-7a5122db-425b-4bc9-85bb-e89c90622e57 tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '957e49fead28485fa4fee73c14041710', 'request_id': 'req-7a5122db-425b-4bc9-85bb-e89c90622e57'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:29.380367 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bf7aae42-c129-4be7-9967-e43904eccda6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3422e322ea884f7e943bc20378e4bbf7', 'request_id': 'req-bf7aae42-c129-4be7-9967-e43904eccda6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:29.384498 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-7a5122db-425b-4bc9-85bb-e89c90622e57 tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] The token cache is being invalidate because project 957e49fead28485fa4fee73c14041710 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:28:29.385084 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-7a5122db-425b-4bc9-85bb-e89c90622e57 tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:29.386041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-7a5122db-425b-4bc9-85bb-e89c90622e57 tempest-VolumeQuotaClassesTest-107834598 tempest-VolumeQuotaClassesTest-107834598-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:29.388290 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 899/1803] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:28:29 2022] DELETE /identity/v3/projects/957e49fead28485fa4fee73c14041710 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:28:29.409606 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bf7aae42-c129-4be7-9967-e43904eccda6 admin admin] The token cache is being invalidate because project 3422e322ea884f7e943bc20378e4bbf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:28:29.409606 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bf7aae42-c129-4be7-9967-e43904eccda6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:29.411449 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bf7aae42-c129-4be7-9967-e43904eccda6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:29.414862 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 905/1804] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:28:28 2022] DELETE /identity/v3/projects/3422e322ea884f7e943bc20378e4bbf7 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:29.420590 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5dda8c-fbf9-4b78-86d6-7ffaf20672d8 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:29.422545 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5dda8c-fbf9-4b78-86d6-7ffaf20672d8 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:29.422987 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5dda8c-fbf9-4b78-86d6-7ffaf20672d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:29.470661 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-6d5dda8c-fbf9-4b78-86d6-7ffaf20672d8 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:29.473024 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6d5dda8c-fbf9-4b78-86d6-7ffaf20672d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:29.673017 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 900/1805] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:29.674904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2afd7-b6c0-4fc8-895c-2a4603f2839a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:29.675193 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2afd7-b6c0-4fc8-895c-2a4603f2839a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:29.675472 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2afd7-b6c0-4fc8-895c-2a4603f2839a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:29.720876 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-33a2afd7-b6c0-4fc8-895c-2a4603f2839a None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:29.727915 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-33a2afd7-b6c0-4fc8-895c-2a4603f2839a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:29.745823 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 906/1806] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:29.750958 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5dda8c-fbf9-4b78-86d6-7ffaf20672d8 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:29.769393 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5dda8c-fbf9-4b78-86d6-7ffaf20672d8 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:29.787586 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dbec6f-fa2f-45c0-9003-c66ab8838370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:29.788450 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbec6f-fa2f-45c0-9003-c66ab8838370 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:29.788889 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbec6f-fa2f-45c0-9003-c66ab8838370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:29.789808 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbec6f-fa2f-45c0-9003-c66ab8838370 admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:29.792783 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dbec6f-fa2f-45c0-9003-c66ab8838370 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:28:29.793280 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dbec6f-fa2f-45c0-9003-c66ab8838370 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:29.800152 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dbec6f-fa2f-45c0-9003-c66ab8838370 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:29.850722 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 901/1807] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:28:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:28:29.854755 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2afd7-b6c0-4fc8-895c-2a4603f2839a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:29.871556 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2afd7-b6c0-4fc8-895c-2a4603f2839a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:29.889967 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5114a6-ef39-45e8-a8cf-ad74bd5c77be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:29.890793 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5114a6-ef39-45e8-a8cf-ad74bd5c77be admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:29.890985 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5114a6-ef39-45e8-a8cf-ad74bd5c77be admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:29.891266 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5114a6-ef39-45e8-a8cf-ad74bd5c77be admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:29.895132 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5114a6-ef39-45e8-a8cf-ad74bd5c77be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:29.906006 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5114a6-ef39-45e8-a8cf-ad74bd5c77be admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:29.943211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dbec6f-fa2f-45c0-9003-c66ab8838370 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:29.946016 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-35dbec6f-fa2f-45c0-9003-c66ab8838370 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:29.976684 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dbec6f-fa2f-45c0-9003-c66ab8838370 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:29.995875 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ec69b-246b-4238-85b8-7b3356b52795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:29.997142 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ec69b-246b-4238-85b8-7b3356b52795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:29.997493 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ec69b-246b-4238-85b8-7b3356b52795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:29.997773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ec69b-246b-4238-85b8-7b3356b52795 admin admin] PATH_INFO: `/v3/users/644451c216c74e818005e66122d79a00` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:30.000755 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947ec69b-246b-4238-85b8-7b3356b52795 admin admin] RBAC: Authorizing `identity:delete_user(user_id=644451c216c74e818005e66122d79a00)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:30.007499 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947ec69b-246b-4238-85b8-7b3356b52795 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:30.035330 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 907/1808] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:28:29 2022] POST /identity/v3/projects => generated 383 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:30.035330 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c42219d-5671-44e6-936d-6f6a7f96ad46 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:30.035800 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c42219d-5671-44e6-936d-6f6a7f96ad46 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:30.035800 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c42219d-5671-44e6-936d-6f6a7f96ad46 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:30.055906 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 908/1809] 10.222.0.26 () {54 vars in 921 bytes} [Fri Sep 9 20:28:30 2022] GET /identity => generated 270 bytes in 24 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:28:30.058081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c42219d-5671-44e6-936d-6f6a7f96ad46 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:30.151033 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-947ec69b-246b-4238-85b8-7b3356b52795 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '644451c216c74e818005e66122d79a00'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:30.200466 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-947ec69b-246b-4238-85b8-7b3356b52795 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '644451c216c74e818005e66122d79a00'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:30.226198 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-947ec69b-246b-4238-85b8-7b3356b52795 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '644451c216c74e818005e66122d79a00'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:30.247461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-947ec69b-246b-4238-85b8-7b3356b52795 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '644451c216c74e818005e66122d79a00'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:30.248124 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-947ec69b-246b-4238-85b8-7b3356b52795 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '644451c216c74e818005e66122d79a00'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:30.260733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 902/1810] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:28:29 2022] DELETE /identity/v3/users/644451c216c74e818005e66122d79a00 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:28:30.262104 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbddfed4-b1c6-4d89-aa4c-2a9a47a1b18c None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:30.262377 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbddfed4-b1c6-4d89-aa4c-2a9a47a1b18c None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:30.262654 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbddfed4-b1c6-4d89-aa4c-2a9a47a1b18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:30.275510 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c42219d-5671-44e6-936d-6f6a7f96ad46 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:30.327977 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-fbddfed4-b1c6-4d89-aa4c-2a9a47a1b18c None None] MFA Rules not processed for user `87ece6128f9245f7b49f3e7a9faaed22`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:30.333288 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-fbddfed4-b1c6-4d89-aa4c-2a9a47a1b18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:30.405581 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:30.477645 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 903/1811] 10.222.0.26 () {58 vars in 1018 bytes} [Fri Sep 9 20:28:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:30.481397 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6cbaff-be47-43dc-9207-f11c141a8426 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:30.483728 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6cbaff-be47-43dc-9207-f11c141a8426 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:30.484060 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6cbaff-be47-43dc-9207-f11c141a8426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:30.501967 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182664ae-d974-478f-946b-a40cc3239f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:30.502669 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182664ae-d974-478f-946b-a40cc3239f19 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:30.502954 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182664ae-d974-478f-946b-a40cc3239f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:30.503227 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182664ae-d974-478f-946b-a40cc3239f19 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:30.503911 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182664ae-d974-478f-946b-a40cc3239f19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:30.518181 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182664ae-d974-478f-946b-a40cc3239f19 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:30.533062 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-3e6cbaff-be47-43dc-9207-f11c141a8426 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:30.534390 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-3e6cbaff-be47-43dc-9207-f11c141a8426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:30.546785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 904/1812] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:30.551132 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a7dbd-95e6-4d55-96b8-552938b2a733 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:30.551132 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a7dbd-95e6-4d55-96b8-552938b2a733 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:30.551132 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a7dbd-95e6-4d55-96b8-552938b2a733 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:30.551132 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 905/1813] 10.222.0.26 () {54 vars in 921 bytes} [Fri Sep 9 20:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:28:30.561107 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a7dbd-95e6-4d55-96b8-552938b2a733 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:30.563033 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a98a7dbd-95e6-4d55-96b8-552938b2a733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:30.608431 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 909/1814] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:28:30 2022] POST /identity/v3/users => generated 292 bytes in 552 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:30.614276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a52230-a538-4ce2-aa75-fada82004edb None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:30.614595 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a52230-a538-4ce2-aa75-fada82004edb None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:30.615064 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a52230-a538-4ce2-aa75-fada82004edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:30.674377 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:30.706277 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f1a52230-a538-4ce2-aa75-fada82004edb None None] MFA Rules not processed for user `87ece6128f9245f7b49f3e7a9faaed22`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:30.710752 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f1a52230-a538-4ce2-aa75-fada82004edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:30.727103 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 910/1815] 10.222.0.26 () {58 vars in 1018 bytes} [Fri Sep 9 20:28:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:30.728877 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a52230-a538-4ce2-aa75-fada82004edb None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:30.730616 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f1a52230-a538-4ce2-aa75-fada82004edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:30.771298 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a7dbd-95e6-4d55-96b8-552938b2a733 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:30.777210 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a52230-a538-4ce2-aa75-fada82004edb None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:30.785736 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ee641-2dee-40fb-aefc-31ea6b1506cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:30.786584 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ee641-2dee-40fb-aefc-31ea6b1506cd service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:30.786853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ee641-2dee-40fb-aefc-31ea6b1506cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:30.787125 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ee641-2dee-40fb-aefc-31ea6b1506cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:30.792401 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6353e4-5a7a-4934-aa30-aa7c6cd69689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:30.793187 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6353e4-5a7a-4934-aa30-aa7c6cd69689 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:30.797978 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6353e4-5a7a-4934-aa30-aa7c6cd69689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:30.798493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6353e4-5a7a-4934-aa30-aa7c6cd69689 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:30.799415 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6353e4-5a7a-4934-aa30-aa7c6cd69689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:30.801373 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6353e4-5a7a-4934-aa30-aa7c6cd69689 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:30.808334 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ee641-2dee-40fb-aefc-31ea6b1506cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:30.813792 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ee641-2dee-40fb-aefc-31ea6b1506cd service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:30.827602 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 911/1816] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:30 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:30.832653 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6353e4-5a7a-4934-aa30-aa7c6cd69689 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:30.842025 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 906/1817] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:30.849198 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6353e4-5a7a-4934-aa30-aa7c6cd69689 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:30.874638 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb723e-64d0-4afa-b28e-b8cf40ec195b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:30.882367 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb723e-64d0-4afa-b28e-b8cf40ec195b admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:30.882723 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb723e-64d0-4afa-b28e-b8cf40ec195b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:30.883043 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb723e-64d0-4afa-b28e-b8cf40ec195b admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:30.888604 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bb723e-64d0-4afa-b28e-b8cf40ec195b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:30.894335 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bb723e-64d0-4afa-b28e-b8cf40ec195b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:30.970119 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 912/1818] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:30 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:28:30.974031 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ee641-2dee-40fb-aefc-31ea6b1506cd service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:31.007866 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ee641-2dee-40fb-aefc-31ea6b1506cd service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:31.030917 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd72c5-fd3e-43e0-a800-7f5beb344f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:31.031681 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd72c5-fd3e-43e0-a800-7f5beb344f18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:31.033802 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd72c5-fd3e-43e0-a800-7f5beb344f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:31.034762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd72c5-fd3e-43e0-a800-7f5beb344f18 admin admin] PATH_INFO: `/v3/projects/291777971b844cecae4de7166c0c0b24/users/4ea0d428a7514902a1680c8149597383/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:31.108106 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd72c5-fd3e-43e0-a800-7f5beb344f18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=291777971b844cecae4de7166c0c0b24, user_id=4ea0d428a7514902a1680c8149597383, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:31.127078 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd72c5-fd3e-43e0-a800-7f5beb344f18 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:31.152622 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb723e-64d0-4afa-b28e-b8cf40ec195b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:31.154480 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-61bb723e-64d0-4afa-b28e-b8cf40ec195b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:31.199419 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 907/1819] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:30 2022] PUT /identity/v3/projects/291777971b844cecae4de7166c0c0b24/users/4ea0d428a7514902a1680c8149597383/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:31.204738 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb723e-64d0-4afa-b28e-b8cf40ec195b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:31.210913 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270df667-30c9-4332-9d2e-673a60080fe5 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:31.211476 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270df667-30c9-4332-9d2e-673a60080fe5 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:31.212063 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270df667-30c9-4332-9d2e-673a60080fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:31.320057 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-270df667-30c9-4332-9d2e-673a60080fe5 None None] MFA Rules not processed for user `4ea0d428a7514902a1680c8149597383`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:31.325536 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-270df667-30c9-4332-9d2e-673a60080fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:31.562273 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080683a7-f264-4c94-8848-0add5b0bbc81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:31.564776 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080683a7-f264-4c94-8848-0add5b0bbc81 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:31.568051 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080683a7-f264-4c94-8848-0add5b0bbc81 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:31.568471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080683a7-f264-4c94-8848-0add5b0bbc81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:31.593330 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 908/1820] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:31.594233 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270df667-30c9-4332-9d2e-673a60080fe5 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:31.598043 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080683a7-f264-4c94-8848-0add5b0bbc81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:31.600533 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080683a7-f264-4c94-8848-0add5b0bbc81 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:31.609261 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-270df667-30c9-4332-9d2e-673a60080fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:31.678058 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 913/1821] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:31.683692 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080683a7-f264-4c94-8848-0add5b0bbc81 service nova] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:31.691113 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-080683a7-f264-4c94-8848-0add5b0bbc81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:31.694351 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270df667-30c9-4332-9d2e-673a60080fe5 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:31.713639 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f9040e-b4c0-4c8a-a136-1359f3709fa5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:31.714469 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f9040e-b4c0-4c8a-a136-1359f3709fa5 service nova] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:31.714787 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f9040e-b4c0-4c8a-a136-1359f3709fa5 service nova] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:31.717944 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f9040e-b4c0-4c8a-a136-1359f3709fa5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:31.764739 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f9040e-b4c0-4c8a-a136-1359f3709fa5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:31.766393 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f9040e-b4c0-4c8a-a136-1359f3709fa5 service nova] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:31.835938 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 909/1822] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:31.879912 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080683a7-f264-4c94-8848-0add5b0bbc81 service nova] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:31.895824 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b428a93-6d40-4209-b31d-93247388a470 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:31.896514 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b428a93-6d40-4209-b31d-93247388a470 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:31.901623 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b428a93-6d40-4209-b31d-93247388a470 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:31.903737 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b428a93-6d40-4209-b31d-93247388a470 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:31.926673 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b428a93-6d40-4209-b31d-93247388a470 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:31.932409 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b428a93-6d40-4209-b31d-93247388a470 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:31.965009 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 914/1823] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:32.394195 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f9040e-b4c0-4c8a-a136-1359f3709fa5 service nova] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:32.398044 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-43f9040e-b4c0-4c8a-a136-1359f3709fa5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:32.461501 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:32.582987 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f9040e-b4c0-4c8a-a136-1359f3709fa5 service nova] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:32.598701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb46e502-de4d-49b4-98b7-400f6d225678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:32.599812 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46e502-de4d-49b4-98b7-400f6d225678 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:32.600427 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46e502-de4d-49b4-98b7-400f6d225678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:32.600921 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46e502-de4d-49b4-98b7-400f6d225678 admin admin] PATH_INFO: `/v3/projects/44c6b1ae2e8546f8bc57ebcfb259ed32` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:32.602604 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb46e502-de4d-49b4-98b7-400f6d225678 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44c6b1ae2e8546f8bc57ebcfb259ed32)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:32.604907 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb46e502-de4d-49b4-98b7-400f6d225678 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:32.706502 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-bb46e502-de4d-49b4-98b7-400f6d225678 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44c6b1ae2e8546f8bc57ebcfb259ed32', 'request_id': 'req-bb46e502-de4d-49b4-98b7-400f6d225678'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:32.722111 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-bb46e502-de4d-49b4-98b7-400f6d225678 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44c6b1ae2e8546f8bc57ebcfb259ed32', 'request_id': 'req-bb46e502-de4d-49b4-98b7-400f6d225678'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:32.739129 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-bb46e502-de4d-49b4-98b7-400f6d225678 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44c6b1ae2e8546f8bc57ebcfb259ed32', 'request_id': 'req-bb46e502-de4d-49b4-98b7-400f6d225678'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:32.770599 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-bb46e502-de4d-49b4-98b7-400f6d225678 admin admin] The token cache is being invalidate because project 44c6b1ae2e8546f8bc57ebcfb259ed32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:28:32.771059 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-bb46e502-de4d-49b4-98b7-400f6d225678 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:32.773960 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-bb46e502-de4d-49b4-98b7-400f6d225678 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:32.781976 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 910/1824] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:28:32 2022] DELETE /identity/v3/projects/44c6b1ae2e8546f8bc57ebcfb259ed32 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:32.783611 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b428a93-6d40-4209-b31d-93247388a470 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:32.800218 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b428a93-6d40-4209-b31d-93247388a470 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:32.938654 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6bfc7c-0bd0-4b37-8813-291b730e5c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:32.941573 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6bfc7c-0bd0-4b37-8813-291b730e5c29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:32.942982 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6bfc7c-0bd0-4b37-8813-291b730e5c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:32.946569 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6bfc7c-0bd0-4b37-8813-291b730e5c29 admin admin] PATH_INFO: `/v3/users/770fff565e854632af1cdfb281fc1575` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:32.948779 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6bfc7c-0bd0-4b37-8813-291b730e5c29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=770fff565e854632af1cdfb281fc1575)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:32.952353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6bfc7c-0bd0-4b37-8813-291b730e5c29 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:33.008711 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:33.174775 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5f6bfc7c-0bd0-4b37-8813-291b730e5c29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '770fff565e854632af1cdfb281fc1575'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:33.212763 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5f6bfc7c-0bd0-4b37-8813-291b730e5c29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '770fff565e854632af1cdfb281fc1575'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:33.258619 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5f6bfc7c-0bd0-4b37-8813-291b730e5c29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '770fff565e854632af1cdfb281fc1575'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:33.276065 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5f6bfc7c-0bd0-4b37-8813-291b730e5c29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '770fff565e854632af1cdfb281fc1575'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:33.276834 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5f6bfc7c-0bd0-4b37-8813-291b730e5c29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '770fff565e854632af1cdfb281fc1575'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:33.288172 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 915/1825] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:28:32 2022] DELETE /identity/v3/users/770fff565e854632af1cdfb281fc1575 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:33.907040 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb46e502-de4d-49b4-98b7-400f6d225678 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:33.908482 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-bb46e502-de4d-49b4-98b7-400f6d225678 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:33.928619 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb46e502-de4d-49b4-98b7-400f6d225678 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:34.062601 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0345c24c-20c1-4136-8759-7f437fbf3414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:34.063738 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345c24c-20c1-4136-8759-7f437fbf3414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:34.064228 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345c24c-20c1-4136-8759-7f437fbf3414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:34.064878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345c24c-20c1-4136-8759-7f437fbf3414 admin admin] PATH_INFO: `/v3/projects/1f93ddca81f54b3d8d48de887071e624` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:34.066609 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0345c24c-20c1-4136-8759-7f437fbf3414 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f93ddca81f54b3d8d48de887071e624)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:34.069909 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0345c24c-20c1-4136-8759-7f437fbf3414 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:34.161931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:34.163371 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-0345c24c-20c1-4136-8759-7f437fbf3414 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f93ddca81f54b3d8d48de887071e624', 'request_id': 'req-0345c24c-20c1-4136-8759-7f437fbf3414'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:34.185342 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-0345c24c-20c1-4136-8759-7f437fbf3414 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f93ddca81f54b3d8d48de887071e624', 'request_id': 'req-0345c24c-20c1-4136-8759-7f437fbf3414'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:34.198208 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-0345c24c-20c1-4136-8759-7f437fbf3414 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f93ddca81f54b3d8d48de887071e624', 'request_id': 'req-0345c24c-20c1-4136-8759-7f437fbf3414'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:34.214447 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-0345c24c-20c1-4136-8759-7f437fbf3414 admin admin] The token cache is being invalidate because project 1f93ddca81f54b3d8d48de887071e624 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:28:34.214812 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-0345c24c-20c1-4136-8759-7f437fbf3414 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:34.215463 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-0345c24c-20c1-4136-8759-7f437fbf3414 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:34.220720 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 911/1826] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:28:33 2022] DELETE /identity/v3/projects/1f93ddca81f54b3d8d48de887071e624 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:34.222564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4be23-e755-4a13-804a-63ebbe0f1246 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:34.223438 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4be23-e755-4a13-804a-63ebbe0f1246 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:34.223713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4be23-e755-4a13-804a-63ebbe0f1246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:34.265087 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-09b4be23-e755-4a13-804a-63ebbe0f1246 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:34.266818 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-09b4be23-e755-4a13-804a-63ebbe0f1246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:34.358817 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 916/1827] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:28:34.366949 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce8e17-69ba-4b86-a05d-62ec4efb27e6 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:34.367408 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce8e17-69ba-4b86-a05d-62ec4efb27e6 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:34.367809 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce8e17-69ba-4b86-a05d-62ec4efb27e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:34.459098 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-f2ce8e17-69ba-4b86-a05d-62ec4efb27e6 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:34.462221 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-f2ce8e17-69ba-4b86-a05d-62ec4efb27e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:34.482828 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 912/1828] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:34.485343 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b4be23-e755-4a13-804a-63ebbe0f1246 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:34.496672 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b4be23-e755-4a13-804a-63ebbe0f1246 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:34.503753 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b1da9-e232-46ea-af83-f98d9cc1f914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:34.504383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b1da9-e232-46ea-af83-f98d9cc1f914 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:34.504661 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b1da9-e232-46ea-af83-f98d9cc1f914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:34.504985 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b1da9-e232-46ea-af83-f98d9cc1f914 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:34.505822 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b1da9-e232-46ea-af83-f98d9cc1f914 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:28:34.506264 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b1da9-e232-46ea-af83-f98d9cc1f914 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:34.508213 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b1da9-e232-46ea-af83-f98d9cc1f914 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:34.527115 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 917/1829] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:28:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:28:34.531729 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce8e17-69ba-4b86-a05d-62ec4efb27e6 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:34.544018 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce8e17-69ba-4b86-a05d-62ec4efb27e6 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:34.552860 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e415004-945d-44e2-8811-cc9417664325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:34.553690 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e415004-945d-44e2-8811-cc9417664325 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:34.553794 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e415004-945d-44e2-8811-cc9417664325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:34.554070 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e415004-945d-44e2-8811-cc9417664325 admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:34.554728 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e415004-945d-44e2-8811-cc9417664325 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:34.557346 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e415004-945d-44e2-8811-cc9417664325 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:34.626965 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 913/1830] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:28:34 2022] POST /identity/v3/projects => generated 391 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:34.630963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b1da9-e232-46ea-af83-f98d9cc1f914 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:34.645412 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b1da9-e232-46ea-af83-f98d9cc1f914 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:34.743433 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d0556-242a-473e-9829-ead87e23c957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:34.744065 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d0556-242a-473e-9829-ead87e23c957 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:34.744345 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d0556-242a-473e-9829-ead87e23c957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:34.744623 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d0556-242a-473e-9829-ead87e23c957 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:34.745298 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266d0556-242a-473e-9829-ead87e23c957 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:34.747521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266d0556-242a-473e-9829-ead87e23c957 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:34.787951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 918/1831] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:28:34 2022] POST /identity/v3/users => generated 296 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:34.792526 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e415004-945d-44e2-8811-cc9417664325 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:34.811396 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e415004-945d-44e2-8811-cc9417664325 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:34.816649 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228b856-64fe-4461-ade1-080dadf7a051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:34.817378 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228b856-64fe-4461-ade1-080dadf7a051 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:34.817757 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228b856-64fe-4461-ade1-080dadf7a051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:34.818137 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228b856-64fe-4461-ade1-080dadf7a051 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:34.818891 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c228b856-64fe-4461-ade1-080dadf7a051 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:34.820614 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c228b856-64fe-4461-ade1-080dadf7a051 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:34.831658 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 914/1832] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:34.835599 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d0556-242a-473e-9829-ead87e23c957 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:34.841144 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:34.848244 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d0556-242a-473e-9829-ead87e23c957 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:34.855618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442f632-2670-43be-846f-6f9c344f71b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:34.856600 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442f632-2670-43be-846f-6f9c344f71b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:34.857066 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442f632-2670-43be-846f-6f9c344f71b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:34.858142 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442f632-2670-43be-846f-6f9c344f71b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:34.858983 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6442f632-2670-43be-846f-6f9c344f71b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:34.860922 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6442f632-2670-43be-846f-6f9c344f71b7 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:34.875449 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 919/1833] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:28:34.880300 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228b856-64fe-4461-ade1-080dadf7a051 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:34.894755 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228b856-64fe-4461-ade1-080dadf7a051 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:34.900912 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e3ad75-3507-48b0-ae4a-eb3a23940c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:34.901647 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3ad75-3507-48b0-ae4a-eb3a23940c60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:34.901928 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3ad75-3507-48b0-ae4a-eb3a23940c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:34.902198 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3ad75-3507-48b0-ae4a-eb3a23940c60 admin admin] PATH_INFO: `/v3/projects/3ee13e693f104220a0ca85bcf0b2cfd8/users/74ca3549394148c7ae20ba724e608a38/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:34.919601 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e3ad75-3507-48b0-ae4a-eb3a23940c60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ee13e693f104220a0ca85bcf0b2cfd8, user_id=74ca3549394148c7ae20ba724e608a38, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:34.931823 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e3ad75-3507-48b0-ae4a-eb3a23940c60 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:34.948927 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 915/1834] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:34 2022] PUT /identity/v3/projects/3ee13e693f104220a0ca85bcf0b2cfd8/users/74ca3549394148c7ae20ba724e608a38/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:34.960807 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0df19b-3026-4543-ad8d-814c3c8c1794 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:34.961096 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0df19b-3026-4543-ad8d-814c3c8c1794 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:34.961414 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0df19b-3026-4543-ad8d-814c3c8c1794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:34.992394 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-fd0df19b-3026-4543-ad8d-814c3c8c1794 None None] MFA Rules not processed for user `74ca3549394148c7ae20ba724e608a38`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:34.993759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-fd0df19b-3026-4543-ad8d-814c3c8c1794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:35.062432 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 920/1835] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:34 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:35.067007 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e3ad75-3507-48b0-ae4a-eb3a23940c60 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:35.138344 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e3ad75-3507-48b0-ae4a-eb3a23940c60 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:35.143582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c416b28a-746c-4bc4-8891-674bf999a779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:35.144193 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416b28a-746c-4bc4-8891-674bf999a779 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:35.144463 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416b28a-746c-4bc4-8891-674bf999a779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:35.144729 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416b28a-746c-4bc4-8891-674bf999a779 admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:35.145407 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c416b28a-746c-4bc4-8891-674bf999a779 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:35.147312 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c416b28a-746c-4bc4-8891-674bf999a779 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:35.173947 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 916/1836] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:28:35 2022] POST /identity/v3/projects => generated 391 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:35.178302 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0df19b-3026-4543-ad8d-814c3c8c1794 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:35.189443 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0df19b-3026-4543-ad8d-814c3c8c1794 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:35.244066 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8967589-954e-4f5b-9d77-f02e60f0334b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:35.244781 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8967589-954e-4f5b-9d77-f02e60f0334b admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:35.244933 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8967589-954e-4f5b-9d77-f02e60f0334b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:35.245206 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8967589-954e-4f5b-9d77-f02e60f0334b admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:35.245854 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8967589-954e-4f5b-9d77-f02e60f0334b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:35.247746 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8967589-954e-4f5b-9d77-f02e60f0334b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:35.268029 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 921/1837] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:28:35 2022] POST /identity/v3/users => generated 302 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:28:35.272152 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c416b28a-746c-4bc4-8891-674bf999a779 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:35.282744 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c416b28a-746c-4bc4-8891-674bf999a779 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:35.287151 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d714ea-ee95-4d82-b36e-57af9051ef8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:35.287738 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d714ea-ee95-4d82-b36e-57af9051ef8c admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:35.288014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d714ea-ee95-4d82-b36e-57af9051ef8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:35.288287 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d714ea-ee95-4d82-b36e-57af9051ef8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:35.288918 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d714ea-ee95-4d82-b36e-57af9051ef8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:35.290533 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d714ea-ee95-4d82-b36e-57af9051ef8c admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:35.301234 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 917/1838] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:35.305449 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8967589-954e-4f5b-9d77-f02e60f0334b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:35.315796 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8967589-954e-4f5b-9d77-f02e60f0334b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:35.320356 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcdbe09-f276-41fb-8d86-2a6b85fc25f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:35.321006 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdbe09-f276-41fb-8d86-2a6b85fc25f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:35.321311 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdbe09-f276-41fb-8d86-2a6b85fc25f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:35.321581 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdbe09-f276-41fb-8d86-2a6b85fc25f7 admin admin] PATH_INFO: `/v3/projects/51a0227cbaca45a8a62162defbd418d1/users/00968a8a5f2f4fb18c1b4f8a1c8c6865/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:35.337552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcdbe09-f276-41fb-8d86-2a6b85fc25f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51a0227cbaca45a8a62162defbd418d1, user_id=00968a8a5f2f4fb18c1b4f8a1c8c6865, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:35.340156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcdbe09-f276-41fb-8d86-2a6b85fc25f7 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:35.353579 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 922/1839] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:35 2022] PUT /identity/v3/projects/51a0227cbaca45a8a62162defbd418d1/users/00968a8a5f2f4fb18c1b4f8a1c8c6865/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:35.361350 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d714ea-ee95-4d82-b36e-57af9051ef8c admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:35.371319 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d714ea-ee95-4d82-b36e-57af9051ef8c admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:35.414130 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:35.428645 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb4c5a5-a34b-4087-9700-7a341cbfca88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:35.429275 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb4c5a5-a34b-4087-9700-7a341cbfca88 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:35.429560 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb4c5a5-a34b-4087-9700-7a341cbfca88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:35.429834 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb4c5a5-a34b-4087-9700-7a341cbfca88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:35.430474 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb4c5a5-a34b-4087-9700-7a341cbfca88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:35.432080 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb4c5a5-a34b-4087-9700-7a341cbfca88 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:35.441933 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 918/1840] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:35 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:35.445995 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcdbe09-f276-41fb-8d86-2a6b85fc25f7 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:35.456230 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcdbe09-f276-41fb-8d86-2a6b85fc25f7 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:35.462471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b7272-26a0-477c-81a2-7ddb000b5a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:35.463151 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b7272-26a0-477c-81a2-7ddb000b5a0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:35.463427 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b7272-26a0-477c-81a2-7ddb000b5a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:35.463698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b7272-26a0-477c-81a2-7ddb000b5a0e admin admin] PATH_INFO: `/v3/projects/51a0227cbaca45a8a62162defbd418d1/users/00968a8a5f2f4fb18c1b4f8a1c8c6865/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:35.465838 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535b7272-26a0-477c-81a2-7ddb000b5a0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=51a0227cbaca45a8a62162defbd418d1, user_id=00968a8a5f2f4fb18c1b4f8a1c8c6865, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:35.469505 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535b7272-26a0-477c-81a2-7ddb000b5a0e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:35.493475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 923/1841] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:35 2022] PUT /identity/v3/projects/51a0227cbaca45a8a62162defbd418d1/users/00968a8a5f2f4fb18c1b4f8a1c8c6865/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:35.495373 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eafddc-a931-4220-a0cd-8a594f64cd84 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:35.496395 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eafddc-a931-4220-a0cd-8a594f64cd84 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:35.496894 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eafddc-a931-4220-a0cd-8a594f64cd84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:35.527951 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-d2eafddc-a931-4220-a0cd-8a594f64cd84 None None] MFA Rules not processed for user `00968a8a5f2f4fb18c1b4f8a1c8c6865`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:35.529169 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-d2eafddc-a931-4220-a0cd-8a594f64cd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:35.602876 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 919/1842] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:35 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:35.616539 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b7272-26a0-477c-81a2-7ddb000b5a0e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:35.618241 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-535b7272-26a0-477c-81a2-7ddb000b5a0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:35.655167 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:35.679947 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b7272-26a0-477c-81a2-7ddb000b5a0e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:35.683604 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb0ee1-2c17-4d08-a87c-360c5a0b3db7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:35.684214 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb0ee1-2c17-4d08-a87c-360c5a0b3db7 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:35.684494 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb0ee1-2c17-4d08-a87c-360c5a0b3db7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:35.685112 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb0ee1-2c17-4d08-a87c-360c5a0b3db7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:35.695918 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb0ee1-2c17-4d08-a87c-360c5a0b3db7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:35.697477 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb0ee1-2c17-4d08-a87c-360c5a0b3db7 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:35.707204 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 924/1843] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:35 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:35.757176 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eafddc-a931-4220-a0cd-8a594f64cd84 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:35.758679 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-d2eafddc-a931-4220-a0cd-8a594f64cd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:35.771316 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eafddc-a931-4220-a0cd-8a594f64cd84 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:35.774947 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb70c1-0006-4b1a-856b-509122c8aa05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:35.775542 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb70c1-0006-4b1a-856b-509122c8aa05 service cinder] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:35.775823 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb70c1-0006-4b1a-856b-509122c8aa05 service cinder] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:35.776093 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb70c1-0006-4b1a-856b-509122c8aa05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:35.834390 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfb70c1-0006-4b1a-856b-509122c8aa05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:35.835921 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfb70c1-0006-4b1a-856b-509122c8aa05 service cinder] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:35.846139 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 920/1844] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:35 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:41.260195 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb0ee1-2c17-4d08-a87c-360c5a0b3db7 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:41.261673 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-15fb0ee1-2c17-4d08-a87c-360c5a0b3db7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:41.393574 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb0ee1-2c17-4d08-a87c-360c5a0b3db7 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:41.401844 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7bec9a-3c67-4f31-9864-8473d7f524ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:41.402593 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7bec9a-3c67-4f31-9864-8473d7f524ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:41.402884 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7bec9a-3c67-4f31-9864-8473d7f524ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:41.403297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7bec9a-3c67-4f31-9864-8473d7f524ad admin admin] PATH_INFO: `/v3/users/65a1c557deb8457ba38878670b96e56d` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:41.407005 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7bec9a-3c67-4f31-9864-8473d7f524ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=65a1c557deb8457ba38878670b96e56d)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:41.409166 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7bec9a-3c67-4f31-9864-8473d7f524ad admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:41.492970 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-8a7bec9a-3c67-4f31-9864-8473d7f524ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65a1c557deb8457ba38878670b96e56d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:41.502815 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-8a7bec9a-3c67-4f31-9864-8473d7f524ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65a1c557deb8457ba38878670b96e56d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:41.520478 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-8a7bec9a-3c67-4f31-9864-8473d7f524ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65a1c557deb8457ba38878670b96e56d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:41.533155 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-8a7bec9a-3c67-4f31-9864-8473d7f524ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65a1c557deb8457ba38878670b96e56d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:41.534178 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-8a7bec9a-3c67-4f31-9864-8473d7f524ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65a1c557deb8457ba38878670b96e56d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:41.552366 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 925/1845] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:28:41 2022] DELETE /identity/v3/users/65a1c557deb8457ba38878670b96e56d => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:41.557816 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2078a-b60f-471d-8545-594baaf916c3 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:41.558355 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2078a-b60f-471d-8545-594baaf916c3 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:41.558355 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2078a-b60f-471d-8545-594baaf916c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:41.589749 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-7bc2078a-b60f-471d-8545-594baaf916c3 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:41.597885 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7bc2078a-b60f-471d-8545-594baaf916c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:41.718871 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 921/1846] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:28:41.735618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7bec9a-3c67-4f31-9864-8473d7f524ad admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:41.739385 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8a7bec9a-3c67-4f31-9864-8473d7f524ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:41.937977 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7bec9a-3c67-4f31-9864-8473d7f524ad admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:41.943042 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998ebf21-1ab3-47a0-84b1-6220c750c952 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:41.943678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ebf21-1ab3-47a0-84b1-6220c750c952 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:41.943957 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ebf21-1ab3-47a0-84b1-6220c750c952 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:41.944230 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ebf21-1ab3-47a0-84b1-6220c750c952 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:41.955845 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998ebf21-1ab3-47a0-84b1-6220c750c952 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:41.957680 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998ebf21-1ab3-47a0-84b1-6220c750c952 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:41.962769 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:41.971161 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 926/1847] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:42.710474 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc2078a-b60f-471d-8545-594baaf916c3 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:42.713075 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7bc2078a-b60f-471d-8545-594baaf916c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:42.727449 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc2078a-b60f-471d-8545-594baaf916c3 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:42.733423 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedcb80f-a3b2-483f-9e11-ae8dfecee507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:42.734505 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcb80f-a3b2-483f-9e11-ae8dfecee507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:42.735037 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcb80f-a3b2-483f-9e11-ae8dfecee507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:42.735407 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcb80f-a3b2-483f-9e11-ae8dfecee507 admin admin] PATH_INFO: `/v3/projects/99832d5ba9b7408b88d9c0f022cd6ef2` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:42.736731 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedcb80f-a3b2-483f-9e11-ae8dfecee507 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99832d5ba9b7408b88d9c0f022cd6ef2)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:42.738971 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedcb80f-a3b2-483f-9e11-ae8dfecee507 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:42.741326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:42.790447 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-aedcb80f-a3b2-483f-9e11-ae8dfecee507 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99832d5ba9b7408b88d9c0f022cd6ef2', 'request_id': 'req-aedcb80f-a3b2-483f-9e11-ae8dfecee507'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:42.798817 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-aedcb80f-a3b2-483f-9e11-ae8dfecee507 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99832d5ba9b7408b88d9c0f022cd6ef2', 'request_id': 'req-aedcb80f-a3b2-483f-9e11-ae8dfecee507'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:42.805224 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-aedcb80f-a3b2-483f-9e11-ae8dfecee507 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99832d5ba9b7408b88d9c0f022cd6ef2', 'request_id': 'req-aedcb80f-a3b2-483f-9e11-ae8dfecee507'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:42.816795 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-aedcb80f-a3b2-483f-9e11-ae8dfecee507 admin admin] The token cache is being invalidate because project 99832d5ba9b7408b88d9c0f022cd6ef2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:28:42.817178 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-aedcb80f-a3b2-483f-9e11-ae8dfecee507 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:42.817939 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-aedcb80f-a3b2-483f-9e11-ae8dfecee507 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:42.819548 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 922/1848] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:28:42 2022] DELETE /identity/v3/projects/99832d5ba9b7408b88d9c0f022cd6ef2 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:42.823632 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998ebf21-1ab3-47a0-84b1-6220c750c952 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:42.833691 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998ebf21-1ab3-47a0-84b1-6220c750c952 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:42.896014 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb8c0d-fc7b-4862-911b-44efcab3d44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:42.896632 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb8c0d-fc7b-4862-911b-44efcab3d44b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:42.897196 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb8c0d-fc7b-4862-911b-44efcab3d44b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:42.897196 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb8c0d-fc7b-4862-911b-44efcab3d44b admin admin] PATH_INFO: `/v3/users/34441ca279ee43919616f3d5eb199344` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:42.898340 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb8c0d-fc7b-4862-911b-44efcab3d44b admin admin] RBAC: Authorizing `identity:delete_user(user_id=34441ca279ee43919616f3d5eb199344)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:42.900274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb8c0d-fc7b-4862-911b-44efcab3d44b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:42.966120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-59eb8c0d-fc7b-4862-911b-44efcab3d44b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34441ca279ee43919616f3d5eb199344'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:42.977806 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-59eb8c0d-fc7b-4862-911b-44efcab3d44b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34441ca279ee43919616f3d5eb199344'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:43.000348 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-59eb8c0d-fc7b-4862-911b-44efcab3d44b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34441ca279ee43919616f3d5eb199344'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:43.009235 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-59eb8c0d-fc7b-4862-911b-44efcab3d44b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34441ca279ee43919616f3d5eb199344'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:43.010255 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-59eb8c0d-fc7b-4862-911b-44efcab3d44b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34441ca279ee43919616f3d5eb199344'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:43.020115 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 927/1849] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:28:42 2022] DELETE /identity/v3/users/34441ca279ee43919616f3d5eb199344 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:43.241376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedcb80f-a3b2-483f-9e11-ae8dfecee507 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:43.243297 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-aedcb80f-a3b2-483f-9e11-ae8dfecee507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:43.283015 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedcb80f-a3b2-483f-9e11-ae8dfecee507 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:43.374986 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7b847-3799-42ee-abbc-574aa0661db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:43.375642 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7b847-3799-42ee-abbc-574aa0661db0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:43.375916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7b847-3799-42ee-abbc-574aa0661db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:43.376187 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7b847-3799-42ee-abbc-574aa0661db0 admin admin] PATH_INFO: `/v3/users/74ca3549394148c7ae20ba724e608a38` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:43.377623 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d7b847-3799-42ee-abbc-574aa0661db0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74ca3549394148c7ae20ba724e608a38)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:43.379566 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d7b847-3799-42ee-abbc-574aa0661db0 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:43.441963 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-d6d7b847-3799-42ee-abbc-574aa0661db0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74ca3549394148c7ae20ba724e608a38'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:43.463354 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-d6d7b847-3799-42ee-abbc-574aa0661db0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74ca3549394148c7ae20ba724e608a38'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:43.477591 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-d6d7b847-3799-42ee-abbc-574aa0661db0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74ca3549394148c7ae20ba724e608a38'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:43.492518 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-d6d7b847-3799-42ee-abbc-574aa0661db0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74ca3549394148c7ae20ba724e608a38'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:43.493391 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-d6d7b847-3799-42ee-abbc-574aa0661db0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74ca3549394148c7ae20ba724e608a38'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:43.516033 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 923/1850] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:28:43 2022] DELETE /identity/v3/users/74ca3549394148c7ae20ba724e608a38 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:43.523781 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30927ae9-b9cc-4adf-81c3-6b5dfccad631 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:43.524299 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30927ae9-b9cc-4adf-81c3-6b5dfccad631 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:43.524823 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30927ae9-b9cc-4adf-81c3-6b5dfccad631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:43.556594 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-30927ae9-b9cc-4adf-81c3-6b5dfccad631 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:43.557938 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-30927ae9-b9cc-4adf-81c3-6b5dfccad631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:43.714316 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 928/1851] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:43.741686 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7b847-3799-42ee-abbc-574aa0661db0 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:43.743408 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-d6d7b847-3799-42ee-abbc-574aa0661db0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:43.771518 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:43.817625 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30927ae9-b9cc-4adf-81c3-6b5dfccad631 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:43.819130 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-30927ae9-b9cc-4adf-81c3-6b5dfccad631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:43.839900 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30927ae9-b9cc-4adf-81c3-6b5dfccad631 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:43.856264 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294976b-0ba2-4cf7-83ab-6cf846c2e01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:43.856938 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294976b-0ba2-4cf7-83ab-6cf846c2e01d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:43.858237 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294976b-0ba2-4cf7-83ab-6cf846c2e01d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:43.858520 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294976b-0ba2-4cf7-83ab-6cf846c2e01d admin admin] PATH_INFO: `/v3/users/0f9e6f3182c8442895558414b5933ee5` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:43.859786 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6294976b-0ba2-4cf7-83ab-6cf846c2e01d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f9e6f3182c8442895558414b5933ee5)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:43.861733 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6294976b-0ba2-4cf7-83ab-6cf846c2e01d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:43.871544 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7b847-3799-42ee-abbc-574aa0661db0 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:43.877812 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42a8008-058f-40a3-acce-ef5815f7f615 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:43.879079 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a8008-058f-40a3-acce-ef5815f7f615 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:43.879429 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a8008-058f-40a3-acce-ef5815f7f615 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:43.880051 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a8008-058f-40a3-acce-ef5815f7f615 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:43.892957 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42a8008-058f-40a3-acce-ef5815f7f615 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:43.894773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42a8008-058f-40a3-acce-ef5815f7f615 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:43.895672 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:43.910233 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 924/1852] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:43.961400 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6294976b-0ba2-4cf7-83ab-6cf846c2e01d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f9e6f3182c8442895558414b5933ee5'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:43.970227 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6294976b-0ba2-4cf7-83ab-6cf846c2e01d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f9e6f3182c8442895558414b5933ee5'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:43.986736 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6294976b-0ba2-4cf7-83ab-6cf846c2e01d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f9e6f3182c8442895558414b5933ee5'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:43.996791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6294976b-0ba2-4cf7-83ab-6cf846c2e01d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f9e6f3182c8442895558414b5933ee5'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:43.997690 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6294976b-0ba2-4cf7-83ab-6cf846c2e01d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f9e6f3182c8442895558414b5933ee5'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:44.008607 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 929/1853] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:28:43 2022] DELETE /identity/v3/users/0f9e6f3182c8442895558414b5933ee5 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:44.015251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232d5ed-5a40-4f26-96e2-dec6f10ed4f6 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:44.015251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232d5ed-5a40-4f26-96e2-dec6f10ed4f6 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:44.015505 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232d5ed-5a40-4f26-96e2-dec6f10ed4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:44.043201 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-5232d5ed-5a40-4f26-96e2-dec6f10ed4f6 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:44.045688 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-5232d5ed-5a40-4f26-96e2-dec6f10ed4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:44.183976 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 925/1854] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:44.286371 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294976b-0ba2-4cf7-83ab-6cf846c2e01d admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:44.287935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6294976b-0ba2-4cf7-83ab-6cf846c2e01d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:44.392470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294976b-0ba2-4cf7-83ab-6cf846c2e01d admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:44.397715 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232d5ed-5a40-4f26-96e2-dec6f10ed4f6 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:44.399136 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-5232d5ed-5a40-4f26-96e2-dec6f10ed4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:44.399587 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7307bd4-2eec-4384-bc8a-105625a61c83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:44.400396 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7307bd4-2eec-4384-bc8a-105625a61c83 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:44.400893 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7307bd4-2eec-4384-bc8a-105625a61c83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:44.401471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7307bd4-2eec-4384-bc8a-105625a61c83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:44.414395 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232d5ed-5a40-4f26-96e2-dec6f10ed4f6 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:44.426616 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7307bd4-2eec-4384-bc8a-105625a61c83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:44.429642 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7307bd4-2eec-4384-bc8a-105625a61c83 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:44.430478 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eef8f7d-5c58-4589-8d73-63ce0c5e9c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:44.432046 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef8f7d-5c58-4589-8d73-63ce0c5e9c4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:44.433321 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef8f7d-5c58-4589-8d73-63ce0c5e9c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:44.434661 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef8f7d-5c58-4589-8d73-63ce0c5e9c4f admin admin] PATH_INFO: `/v3/users/4ea0d428a7514902a1680c8149597383` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:44.439793 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eef8f7d-5c58-4589-8d73-63ce0c5e9c4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ea0d428a7514902a1680c8149597383)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:44.444892 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eef8f7d-5c58-4589-8d73-63ce0c5e9c4f admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:44.460125 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 930/1855] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:44.540813 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6eef8f7d-5c58-4589-8d73-63ce0c5e9c4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ea0d428a7514902a1680c8149597383'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:44.585536 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6eef8f7d-5c58-4589-8d73-63ce0c5e9c4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ea0d428a7514902a1680c8149597383'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:44.601222 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6eef8f7d-5c58-4589-8d73-63ce0c5e9c4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ea0d428a7514902a1680c8149597383'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:44.611876 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6eef8f7d-5c58-4589-8d73-63ce0c5e9c4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ea0d428a7514902a1680c8149597383'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:44.612625 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6eef8f7d-5c58-4589-8d73-63ce0c5e9c4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ea0d428a7514902a1680c8149597383'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:44.621294 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 926/1856] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:28:44 2022] DELETE /identity/v3/users/4ea0d428a7514902a1680c8149597383 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:44.626590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaae2ab-90dd-4d0d-8afc-738da11f7efd None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:44.627041 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaae2ab-90dd-4d0d-8afc-738da11f7efd None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:44.627465 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaae2ab-90dd-4d0d-8afc-738da11f7efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:44.647779 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-aeaae2ab-90dd-4d0d-8afc-738da11f7efd None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:44.648984 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-aeaae2ab-90dd-4d0d-8afc-738da11f7efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:44.739033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 931/1857] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:28:44.772161 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eef8f7d-5c58-4589-8d73-63ce0c5e9c4f admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:44.776438 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6eef8f7d-5c58-4589-8d73-63ce0c5e9c4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:44.818516 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:44.859790 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eef8f7d-5c58-4589-8d73-63ce0c5e9c4f admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:44.865184 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea2c32-e4e5-47e4-ac37-10fd2ad02190 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:44.866381 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea2c32-e4e5-47e4-ac37-10fd2ad02190 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:44.868219 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea2c32-e4e5-47e4-ac37-10fd2ad02190 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:44.868721 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea2c32-e4e5-47e4-ac37-10fd2ad02190 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:44.882361 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ea2c32-e4e5-47e4-ac37-10fd2ad02190 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:44.884803 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaae2ab-90dd-4d0d-8afc-738da11f7efd None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:44.887060 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-aeaae2ab-90dd-4d0d-8afc-738da11f7efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:44.889809 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ea2c32-e4e5-47e4-ac37-10fd2ad02190 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:44.901512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaae2ab-90dd-4d0d-8afc-738da11f7efd None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:44.907361 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce90295-f9e3-457d-9ced-e3d4736adefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:44.908154 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce90295-f9e3-457d-9ced-e3d4736adefd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:44.908516 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce90295-f9e3-457d-9ced-e3d4736adefd admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:44.911622 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce90295-f9e3-457d-9ced-e3d4736adefd admin admin] PATH_INFO: `/v3/projects/37588baaad34495ca5e4adc1e2cbaecc` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:44.911622 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce90295-f9e3-457d-9ced-e3d4736adefd admin admin] RBAC: Authorizing `identity:delete_project(project_id=37588baaad34495ca5e4adc1e2cbaecc)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:44.913366 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 927/1858] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:44.913647 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce90295-f9e3-457d-9ced-e3d4736adefd admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:44.969934 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:44.990974 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7ce90295-f9e3-457d-9ced-e3d4736adefd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37588baaad34495ca5e4adc1e2cbaecc', 'request_id': 'req-7ce90295-f9e3-457d-9ced-e3d4736adefd'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:44.997667 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7ce90295-f9e3-457d-9ced-e3d4736adefd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37588baaad34495ca5e4adc1e2cbaecc', 'request_id': 'req-7ce90295-f9e3-457d-9ced-e3d4736adefd'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:45.006177 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7ce90295-f9e3-457d-9ced-e3d4736adefd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37588baaad34495ca5e4adc1e2cbaecc', 'request_id': 'req-7ce90295-f9e3-457d-9ced-e3d4736adefd'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:45.022039 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7ce90295-f9e3-457d-9ced-e3d4736adefd admin admin] The token cache is being invalidate because project 37588baaad34495ca5e4adc1e2cbaecc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:28:45.022333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7ce90295-f9e3-457d-9ced-e3d4736adefd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:45.023078 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7ce90295-f9e3-457d-9ced-e3d4736adefd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:45.028239 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 932/1859] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:28:44 2022] DELETE /identity/v3/projects/37588baaad34495ca5e4adc1e2cbaecc => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:28:45.031136 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774c494-f883-49a7-891a-1b7f377c2b6d None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:45.031431 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774c494-f883-49a7-891a-1b7f377c2b6d None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:45.031710 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774c494-f883-49a7-891a-1b7f377c2b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:45.064275 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-5774c494-f883-49a7-891a-1b7f377c2b6d None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:45.066214 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-5774c494-f883-49a7-891a-1b7f377c2b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:45.163740 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce90295-f9e3-457d-9ced-e3d4736adefd admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:45.165921 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7ce90295-f9e3-457d-9ced-e3d4736adefd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:45.172722 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 928/1860] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:45.181187 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a237f-95fc-4de2-acfa-99c517fb23b6 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:45.183060 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a237f-95fc-4de2-acfa-99c517fb23b6 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:45.183441 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a237f-95fc-4de2-acfa-99c517fb23b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:45.188153 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce90295-f9e3-457d-9ced-e3d4736adefd admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:45.208701 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978dc51-12fc-4440-b176-7e23bc96bfda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:45.209475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978dc51-12fc-4440-b176-7e23bc96bfda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:45.209766 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978dc51-12fc-4440-b176-7e23bc96bfda admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:45.210045 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978dc51-12fc-4440-b176-7e23bc96bfda admin admin] PATH_INFO: `/v3/projects/3ee13e693f104220a0ca85bcf0b2cfd8` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:45.211226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978dc51-12fc-4440-b176-7e23bc96bfda admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ee13e693f104220a0ca85bcf0b2cfd8)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:45.214902 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978dc51-12fc-4440-b176-7e23bc96bfda admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:45.259143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-199a237f-95fc-4de2-acfa-99c517fb23b6 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:45.260406 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-199a237f-95fc-4de2-acfa-99c517fb23b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:45.283871 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 929/1861] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:28:45.285391 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a237f-95fc-4de2-acfa-99c517fb23b6 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:45.292277 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7978dc51-12fc-4440-b176-7e23bc96bfda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ee13e693f104220a0ca85bcf0b2cfd8', 'request_id': 'req-7978dc51-12fc-4440-b176-7e23bc96bfda'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:45.299871 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a237f-95fc-4de2-acfa-99c517fb23b6 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:45.304395 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7978dc51-12fc-4440-b176-7e23bc96bfda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ee13e693f104220a0ca85bcf0b2cfd8', 'request_id': 'req-7978dc51-12fc-4440-b176-7e23bc96bfda'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:45.313530 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7978dc51-12fc-4440-b176-7e23bc96bfda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ee13e693f104220a0ca85bcf0b2cfd8', 'request_id': 'req-7978dc51-12fc-4440-b176-7e23bc96bfda'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:45.342578 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7978dc51-12fc-4440-b176-7e23bc96bfda admin admin] The token cache is being invalidate because project 3ee13e693f104220a0ca85bcf0b2cfd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:28:45.342917 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7978dc51-12fc-4440-b176-7e23bc96bfda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:45.343840 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7978dc51-12fc-4440-b176-7e23bc96bfda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:45.349876 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 933/1862] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:28:45 2022] DELETE /identity/v3/projects/3ee13e693f104220a0ca85bcf0b2cfd8 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:45.353338 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978dc51-12fc-4440-b176-7e23bc96bfda admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:45.354843 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7978dc51-12fc-4440-b176-7e23bc96bfda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:45.370472 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06af315-7dac-48df-bf91-97fa5f69d4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:45.371762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06af315-7dac-48df-bf91-97fa5f69d4ef admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:45.371762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06af315-7dac-48df-bf91-97fa5f69d4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:45.371762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06af315-7dac-48df-bf91-97fa5f69d4ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:45.372708 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06af315-7dac-48df-bf91-97fa5f69d4ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:28:45.372946 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06af315-7dac-48df-bf91-97fa5f69d4ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:45.375072 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06af315-7dac-48df-bf91-97fa5f69d4ef admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:45.391764 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 930/1863] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:28:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:28:45.391764 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06af315-7dac-48df-bf91-97fa5f69d4ef admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:45.393644 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-e06af315-7dac-48df-bf91-97fa5f69d4ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:45.409411 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06af315-7dac-48df-bf91-97fa5f69d4ef admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:45.415696 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928c6e8-5de7-4d3e-a2fc-95710dfc554d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:45.416400 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928c6e8-5de7-4d3e-a2fc-95710dfc554d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:45.417426 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928c6e8-5de7-4d3e-a2fc-95710dfc554d admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:45.418017 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928c6e8-5de7-4d3e-a2fc-95710dfc554d admin admin] PATH_INFO: `/v3/users/00968a8a5f2f4fb18c1b4f8a1c8c6865` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:45.418975 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2928c6e8-5de7-4d3e-a2fc-95710dfc554d admin admin] RBAC: Authorizing `identity:delete_user(user_id=00968a8a5f2f4fb18c1b4f8a1c8c6865)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:45.421595 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2928c6e8-5de7-4d3e-a2fc-95710dfc554d admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:45.431786 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978dc51-12fc-4440-b176-7e23bc96bfda admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:45.437751 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7c6ff-233b-474b-b8e4-f3b3b7217a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:45.438583 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7c6ff-233b-474b-b8e4-f3b3b7217a6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:45.438932 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7c6ff-233b-474b-b8e4-f3b3b7217a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:45.439284 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7c6ff-233b-474b-b8e4-f3b3b7217a6f admin admin] PATH_INFO: `/v3/projects/8f5e1579b6d3470eaf120e05c9a466dc` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:45.442367 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd7c6ff-233b-474b-b8e4-f3b3b7217a6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f5e1579b6d3470eaf120e05c9a466dc)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:45.444075 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd7c6ff-233b-474b-b8e4-f3b3b7217a6f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:45.477633 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-2928c6e8-5de7-4d3e-a2fc-95710dfc554d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00968a8a5f2f4fb18c1b4f8a1c8c6865'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:45.480037 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:45.496255 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:45.501239 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-2928c6e8-5de7-4d3e-a2fc-95710dfc554d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00968a8a5f2f4fb18c1b4f8a1c8c6865'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:45.506172 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bbd7c6ff-233b-474b-b8e4-f3b3b7217a6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f5e1579b6d3470eaf120e05c9a466dc', 'request_id': 'req-bbd7c6ff-233b-474b-b8e4-f3b3b7217a6f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:45.509675 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-2928c6e8-5de7-4d3e-a2fc-95710dfc554d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00968a8a5f2f4fb18c1b4f8a1c8c6865'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:45.517526 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bbd7c6ff-233b-474b-b8e4-f3b3b7217a6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f5e1579b6d3470eaf120e05c9a466dc', 'request_id': 'req-bbd7c6ff-233b-474b-b8e4-f3b3b7217a6f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:45.519094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-2928c6e8-5de7-4d3e-a2fc-95710dfc554d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00968a8a5f2f4fb18c1b4f8a1c8c6865'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:45.520169 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-2928c6e8-5de7-4d3e-a2fc-95710dfc554d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00968a8a5f2f4fb18c1b4f8a1c8c6865'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:45.524296 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bbd7c6ff-233b-474b-b8e4-f3b3b7217a6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f5e1579b6d3470eaf120e05c9a466dc', 'request_id': 'req-bbd7c6ff-233b-474b-b8e4-f3b3b7217a6f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:45.530401 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 931/1864] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:28:45 2022] DELETE /identity/v3/users/00968a8a5f2f4fb18c1b4f8a1c8c6865 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:45.531434 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928c6e8-5de7-4d3e-a2fc-95710dfc554d admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:45.533618 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-2928c6e8-5de7-4d3e-a2fc-95710dfc554d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:45.539512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bbd7c6ff-233b-474b-b8e4-f3b3b7217a6f admin admin] The token cache is being invalidate because project 8f5e1579b6d3470eaf120e05c9a466dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:28:45.540223 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bbd7c6ff-233b-474b-b8e4-f3b3b7217a6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:45.541015 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bbd7c6ff-233b-474b-b8e4-f3b3b7217a6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:45.543128 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 934/1865] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:28:45 2022] DELETE /identity/v3/projects/8f5e1579b6d3470eaf120e05c9a466dc => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:45.547620 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7c6ff-233b-474b-b8e4-f3b3b7217a6f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:45.549342 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-bbd7c6ff-233b-474b-b8e4-f3b3b7217a6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:45.665392 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928c6e8-5de7-4d3e-a2fc-95710dfc554d admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:45.671329 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c61e7c3-ad0a-4975-b38d-3c0a7592e621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:45.672099 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61e7c3-ad0a-4975-b38d-3c0a7592e621 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:45.672538 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61e7c3-ad0a-4975-b38d-3c0a7592e621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:45.673211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61e7c3-ad0a-4975-b38d-3c0a7592e621 admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:45.674098 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c61e7c3-ad0a-4975-b38d-3c0a7592e621 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:45.676112 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c61e7c3-ad0a-4975-b38d-3c0a7592e621 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:45.699222 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7c6ff-233b-474b-b8e4-f3b3b7217a6f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:45.712580 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6f1ca-0dc2-4170-b1f5-b69a31d98d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:45.713386 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6f1ca-0dc2-4170-b1f5-b69a31d98d11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:45.714401 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6f1ca-0dc2-4170-b1f5-b69a31d98d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:45.714401 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6f1ca-0dc2-4170-b1f5-b69a31d98d11 admin admin] PATH_INFO: `/v3/users/2874613639ab4388ac76c9b29cccb60f` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:45.720588 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d6f1ca-0dc2-4170-b1f5-b69a31d98d11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2874613639ab4388ac76c9b29cccb60f)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:45.724144 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d6f1ca-0dc2-4170-b1f5-b69a31d98d11 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:45.743193 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 932/1866] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:28:45 2022] POST /identity/v3/projects => generated 373 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:45.744412 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c61e7c3-ad0a-4975-b38d-3c0a7592e621 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:45.747651 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-4c61e7c3-ad0a-4975-b38d-3c0a7592e621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:45.869626 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-55d6f1ca-0dc2-4170-b1f5-b69a31d98d11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2874613639ab4388ac76c9b29cccb60f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:45.900457 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-55d6f1ca-0dc2-4170-b1f5-b69a31d98d11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2874613639ab4388ac76c9b29cccb60f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:45.935859 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-55d6f1ca-0dc2-4170-b1f5-b69a31d98d11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2874613639ab4388ac76c9b29cccb60f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:45.946131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c61e7c3-ad0a-4975-b38d-3c0a7592e621 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:45.946741 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-55d6f1ca-0dc2-4170-b1f5-b69a31d98d11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2874613639ab4388ac76c9b29cccb60f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:45.948664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-55d6f1ca-0dc2-4170-b1f5-b69a31d98d11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2874613639ab4388ac76c9b29cccb60f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:45.952690 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83340a71-a7c0-4654-b5f8-7a848ee5e348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:45.953475 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83340a71-a7c0-4654-b5f8-7a848ee5e348 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:45.953830 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83340a71-a7c0-4654-b5f8-7a848ee5e348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:45.954179 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83340a71-a7c0-4654-b5f8-7a848ee5e348 admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:45.954921 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83340a71-a7c0-4654-b5f8-7a848ee5e348 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:45.957073 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83340a71-a7c0-4654-b5f8-7a848ee5e348 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:45.960527 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 935/1867] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:28:45 2022] DELETE /identity/v3/users/2874613639ab4388ac76c9b29cccb60f => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:28:45.991611 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 933/1868] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:28:45 2022] POST /identity/v3/users => generated 287 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:45.995398 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6f1ca-0dc2-4170-b1f5-b69a31d98d11 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:45.997002 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-55d6f1ca-0dc2-4170-b1f5-b69a31d98d11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:46.068554 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:46.090514 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6f1ca-0dc2-4170-b1f5-b69a31d98d11 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:46.097971 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5ed248-b558-4c3b-bd91-979178940b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:46.098617 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ed248-b558-4c3b-bd91-979178940b25 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:46.098899 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ed248-b558-4c3b-bd91-979178940b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:46.099174 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ed248-b558-4c3b-bd91-979178940b25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:46.099833 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5ed248-b558-4c3b-bd91-979178940b25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:46.101683 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5ed248-b558-4c3b-bd91-979178940b25 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:46.117554 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 936/1869] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:46.122006 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83340a71-a7c0-4654-b5f8-7a848ee5e348 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:46.145200 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83340a71-a7c0-4654-b5f8-7a848ee5e348 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:46.152632 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94b2d1-ca1c-46da-83b8-6d9f87b2c718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:46.152632 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94b2d1-ca1c-46da-83b8-6d9f87b2c718 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:46.152632 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94b2d1-ca1c-46da-83b8-6d9f87b2c718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:46.152632 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94b2d1-ca1c-46da-83b8-6d9f87b2c718 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:46.153189 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94b2d1-ca1c-46da-83b8-6d9f87b2c718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:46.154867 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94b2d1-ca1c-46da-83b8-6d9f87b2c718 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:46.180761 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 934/1870] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:46.184475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5ed248-b558-4c3b-bd91-979178940b25 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:46.202203 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5ed248-b558-4c3b-bd91-979178940b25 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:46.208384 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda8364-1fa5-4ef9-a782-84adec6c12e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:46.209429 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda8364-1fa5-4ef9-a782-84adec6c12e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:46.209824 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda8364-1fa5-4ef9-a782-84adec6c12e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:46.210253 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda8364-1fa5-4ef9-a782-84adec6c12e9 admin admin] PATH_INFO: `/v3/projects/8ee2ee965b4d497da9ca22bab80d9211/users/14645a6efb674666a1b9bde5442f6d0f/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:46.233165 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda8364-1fa5-4ef9-a782-84adec6c12e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ee2ee965b4d497da9ca22bab80d9211, user_id=14645a6efb674666a1b9bde5442f6d0f, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:46.236224 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda8364-1fa5-4ef9-a782-84adec6c12e9 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:46.260900 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 937/1871] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:46 2022] PUT /identity/v3/projects/8ee2ee965b4d497da9ca22bab80d9211/users/14645a6efb674666a1b9bde5442f6d0f/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:46.271033 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fca4c8-f343-4689-8aaf-ed655a5bffb8 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:46.271589 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fca4c8-f343-4689-8aaf-ed655a5bffb8 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:46.273051 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fca4c8-f343-4689-8aaf-ed655a5bffb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:46.313295 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe557d-597e-4b4f-adf5-43cdc129adfe None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:46.313542 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe557d-597e-4b4f-adf5-43cdc129adfe None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:46.313778 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe557d-597e-4b4f-adf5-43cdc129adfe None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:46.315093 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 938/1872] 10.222.0.26 () {54 vars in 921 bytes} [Fri Sep 9 20:28:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:28:46.322851 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-21fca4c8-f343-4689-8aaf-ed655a5bffb8 None None] MFA Rules not processed for user `14645a6efb674666a1b9bde5442f6d0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:46.324230 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-21fca4c8-f343-4689-8aaf-ed655a5bffb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:46.331168 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c97c0-06e8-4104-bc41-974e546f6621 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:46.331400 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c97c0-06e8-4104-bc41-974e546f6621 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:46.331798 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c97c0-06e8-4104-bc41-974e546f6621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:46.375969 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0a0c97c0-06e8-4104-bc41-974e546f6621 None None] MFA Rules not processed for user `87ece6128f9245f7b49f3e7a9faaed22`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:46.377639 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0a0c97c0-06e8-4104-bc41-974e546f6621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:46.381626 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:46.522707 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 935/1873] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:46.523432 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fca4c8-f343-4689-8aaf-ed655a5bffb8 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:46.525296 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-21fca4c8-f343-4689-8aaf-ed655a5bffb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:46.583258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 939/1874] 10.222.0.26 () {58 vars in 1018 bytes} [Fri Sep 9 20:28:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:46.584473 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c97c0-06e8-4104-bc41-974e546f6621 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:46.675815 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fca4c8-f343-4689-8aaf-ed655a5bffb8 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:46.676904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c97c0-06e8-4104-bc41-974e546f6621 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:46.682897 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a9582-241f-488d-af27-d69d956c02e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:46.683540 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656809ad-7ab2-45b7-9cab-62261614a808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:46.683669 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a9582-241f-488d-af27-d69d956c02e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:46.683778 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a9582-241f-488d-af27-d69d956c02e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:46.684051 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a9582-241f-488d-af27-d69d956c02e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:46.684490 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656809ad-7ab2-45b7-9cab-62261614a808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:46.684611 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656809ad-7ab2-45b7-9cab-62261614a808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:46.684872 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656809ad-7ab2-45b7-9cab-62261614a808 admin admin] PATH_INFO: `/v3/projects/291777971b844cecae4de7166c0c0b24` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:46.685065 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3a9582-241f-488d-af27-d69d956c02e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:46.686640 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656809ad-7ab2-45b7-9cab-62261614a808 admin admin] RBAC: Authorizing `identity:delete_project(project_id=291777971b844cecae4de7166c0c0b24)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:46.687849 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3a9582-241f-488d-af27-d69d956c02e0 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:46.688808 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656809ad-7ab2-45b7-9cab-62261614a808 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:46.728265 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 940/1875] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:28:46 2022] POST /identity/v3/projects => generated 371 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:46.729371 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a9582-241f-488d-af27-d69d956c02e0 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:46.731509 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ca3a9582-241f-488d-af27-d69d956c02e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:46.755307 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-656809ad-7ab2-45b7-9cab-62261614a808 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '291777971b844cecae4de7166c0c0b24', 'request_id': 'req-656809ad-7ab2-45b7-9cab-62261614a808'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:46.766452 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-656809ad-7ab2-45b7-9cab-62261614a808 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '291777971b844cecae4de7166c0c0b24', 'request_id': 'req-656809ad-7ab2-45b7-9cab-62261614a808'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:46.776812 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-656809ad-7ab2-45b7-9cab-62261614a808 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '291777971b844cecae4de7166c0c0b24', 'request_id': 'req-656809ad-7ab2-45b7-9cab-62261614a808'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:46.785668 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:46.800907 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-656809ad-7ab2-45b7-9cab-62261614a808 admin admin] The token cache is being invalidate because project 291777971b844cecae4de7166c0c0b24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:28:46.801439 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-656809ad-7ab2-45b7-9cab-62261614a808 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:46.802150 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-656809ad-7ab2-45b7-9cab-62261614a808 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:46.803959 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 936/1876] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:28:46 2022] DELETE /identity/v3/projects/291777971b844cecae4de7166c0c0b24 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:46.808098 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656809ad-7ab2-45b7-9cab-62261614a808 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:46.809624 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-656809ad-7ab2-45b7-9cab-62261614a808 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:46.820226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a9582-241f-488d-af27-d69d956c02e0 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:46.867317 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:46.895099 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656809ad-7ab2-45b7-9cab-62261614a808 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:46.902918 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ec3f9-9377-4bb4-baf0-1d57c1474314 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:46.903640 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ec3f9-9377-4bb4-baf0-1d57c1474314 service nova] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:46.904066 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ec3f9-9377-4bb4-baf0-1d57c1474314 service nova] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:46.904410 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ec3f9-9377-4bb4-baf0-1d57c1474314 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:46.906371 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7d7ec3f9-9377-4bb4-baf0-1d57c1474314 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:46.918277 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca03bcf-93e1-4d19-b64c-9d07387c0fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:46.919042 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca03bcf-93e1-4d19-b64c-9d07387c0fd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:46.919219 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca03bcf-93e1-4d19-b64c-9d07387c0fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:46.920174 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca03bcf-93e1-4d19-b64c-9d07387c0fd4 admin admin] PATH_INFO: `/v3/projects/e85c36d05d3b4bee8645d3d65bf46b2f` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:46.920601 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ec3f9-9377-4bb4-baf0-1d57c1474314 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:46.921439 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca03bcf-93e1-4d19-b64c-9d07387c0fd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e85c36d05d3b4bee8645d3d65bf46b2f)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:46.922367 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ec3f9-9377-4bb4-baf0-1d57c1474314 service nova] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:46.923326 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca03bcf-93e1-4d19-b64c-9d07387c0fd4 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:46.935478 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 937/1877] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:46.936382 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ec3f9-9377-4bb4-baf0-1d57c1474314 service nova] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:46.937869 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7d7ec3f9-9377-4bb4-baf0-1d57c1474314 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:46.952496 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ec3f9-9377-4bb4-baf0-1d57c1474314 service nova] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:46.958287 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2fa2d3-ba14-4560-b840-80e14af0676c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:46.959074 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2fa2d3-ba14-4560-b840-80e14af0676c admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:46.959426 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2fa2d3-ba14-4560-b840-80e14af0676c admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:46.959799 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2fa2d3-ba14-4560-b840-80e14af0676c admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:46.960661 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2fa2d3-ba14-4560-b840-80e14af0676c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:46.963396 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2fa2d3-ba14-4560-b840-80e14af0676c admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:46.984096 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2ca03bcf-93e1-4d19-b64c-9d07387c0fd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e85c36d05d3b4bee8645d3d65bf46b2f', 'request_id': 'req-2ca03bcf-93e1-4d19-b64c-9d07387c0fd4'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:46.991991 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2ca03bcf-93e1-4d19-b64c-9d07387c0fd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e85c36d05d3b4bee8645d3d65bf46b2f', 'request_id': 'req-2ca03bcf-93e1-4d19-b64c-9d07387c0fd4'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:46.994591 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 938/1878] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:28:46 2022] POST /identity/v3/users => generated 292 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:47.000983 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455488f-174a-4c91-9255-1bac90ba9d90 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:47.001333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2ca03bcf-93e1-4d19-b64c-9d07387c0fd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e85c36d05d3b4bee8645d3d65bf46b2f', 'request_id': 'req-2ca03bcf-93e1-4d19-b64c-9d07387c0fd4'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:47.001598 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455488f-174a-4c91-9255-1bac90ba9d90 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:47.001946 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455488f-174a-4c91-9255-1bac90ba9d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:47.024112 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2ca03bcf-93e1-4d19-b64c-9d07387c0fd4 admin admin] The token cache is being invalidate because project e85c36d05d3b4bee8645d3d65bf46b2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:28:47.024699 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2ca03bcf-93e1-4d19-b64c-9d07387c0fd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:47.026076 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2ca03bcf-93e1-4d19-b64c-9d07387c0fd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:47.027022 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-1455488f-174a-4c91-9255-1bac90ba9d90 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:47.028102 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 941/1879] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:28:46 2022] DELETE /identity/v3/projects/e85c36d05d3b4bee8645d3d65bf46b2f => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:47.028642 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-1455488f-174a-4c91-9255-1bac90ba9d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:47.029805 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca03bcf-93e1-4d19-b64c-9d07387c0fd4 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:47.032854 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2ca03bcf-93e1-4d19-b64c-9d07387c0fd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:47.110206 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 939/1880] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:47.111661 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1455488f-174a-4c91-9255-1bac90ba9d90 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:47.118636 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-1455488f-174a-4c91-9255-1bac90ba9d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:47.133439 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca03bcf-93e1-4d19-b64c-9d07387c0fd4 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:47.133791 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1455488f-174a-4c91-9255-1bac90ba9d90 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:47.139207 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676c45a-1a49-4168-a84b-4442fa899439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:47.139862 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d11a1-7576-40a4-b1a7-ed7917284086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:47.140253 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676c45a-1a49-4168-a84b-4442fa899439 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:47.140659 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d11a1-7576-40a4-b1a7-ed7917284086 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:47.140914 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676c45a-1a49-4168-a84b-4442fa899439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:47.141072 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d11a1-7576-40a4-b1a7-ed7917284086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:47.141488 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d11a1-7576-40a4-b1a7-ed7917284086 admin admin] PATH_INFO: `/v3/projects/51a0227cbaca45a8a62162defbd418d1` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:47.141742 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676c45a-1a49-4168-a84b-4442fa899439 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:47.142567 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676c45a-1a49-4168-a84b-4442fa899439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:47.142940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d11a1-7576-40a4-b1a7-ed7917284086 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51a0227cbaca45a8a62162defbd418d1)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:47.146232 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d11a1-7576-40a4-b1a7-ed7917284086 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:47.146386 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676c45a-1a49-4168-a84b-4442fa899439 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:47.165982 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:47.179302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 940/1881] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:47.181678 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed2fbc-6b6a-4bc3-802b-892db7e72b6e None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:47.182097 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed2fbc-6b6a-4bc3-802b-892db7e72b6e None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:47.182486 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed2fbc-6b6a-4bc3-802b-892db7e72b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:47.211459 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-366d11a1-7576-40a4-b1a7-ed7917284086 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51a0227cbaca45a8a62162defbd418d1', 'request_id': 'req-366d11a1-7576-40a4-b1a7-ed7917284086'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:47.216050 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-98ed2fbc-6b6a-4bc3-802b-892db7e72b6e None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:47.218060 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-98ed2fbc-6b6a-4bc3-802b-892db7e72b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:47.219240 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-366d11a1-7576-40a4-b1a7-ed7917284086 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51a0227cbaca45a8a62162defbd418d1', 'request_id': 'req-366d11a1-7576-40a4-b1a7-ed7917284086'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:47.226360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-366d11a1-7576-40a4-b1a7-ed7917284086 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51a0227cbaca45a8a62162defbd418d1', 'request_id': 'req-366d11a1-7576-40a4-b1a7-ed7917284086'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:47.243174 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-366d11a1-7576-40a4-b1a7-ed7917284086 admin admin] The token cache is being invalidate because project 51a0227cbaca45a8a62162defbd418d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:28:47.243629 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-366d11a1-7576-40a4-b1a7-ed7917284086 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:47.244445 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-366d11a1-7576-40a4-b1a7-ed7917284086 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:47.246131 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 942/1882] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:28:47 2022] DELETE /identity/v3/projects/51a0227cbaca45a8a62162defbd418d1 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:47.247420 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf0b4b-64e9-4090-b9c7-517788444a7b None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:47.247803 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf0b4b-64e9-4090-b9c7-517788444a7b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:47.248157 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf0b4b-64e9-4090-b9c7-517788444a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:47.271439 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-19bf0b4b-64e9-4090-b9c7-517788444a7b None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:47.272690 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-19bf0b4b-64e9-4090-b9c7-517788444a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:47.293295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 941/1883] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:47.293950 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed2fbc-6b6a-4bc3-802b-892db7e72b6e None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:47.295300 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-98ed2fbc-6b6a-4bc3-802b-892db7e72b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:47.306843 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed2fbc-6b6a-4bc3-802b-892db7e72b6e None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:47.312794 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f50ff-3de5-4ad4-ab20-82f740af9291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:47.313490 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f50ff-3de5-4ad4-ab20-82f740af9291 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:47.313817 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f50ff-3de5-4ad4-ab20-82f740af9291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:47.314168 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f50ff-3de5-4ad4-ab20-82f740af9291 admin admin] PATH_INFO: `/v3/projects/703cf460339a4aaf8571934915b76ba6/users/5f66c5b0b6dd4c488782a8f1a85fe80b/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:47.329335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f50ff-3de5-4ad4-ab20-82f740af9291 admin admin] RBAC: Authorizing `identity:create_grant(project_id=703cf460339a4aaf8571934915b76ba6, user_id=5f66c5b0b6dd4c488782a8f1a85fe80b, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:47.331975 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f50ff-3de5-4ad4-ab20-82f740af9291 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:47.348721 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 942/1884] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:47 2022] PUT /identity/v3/projects/703cf460339a4aaf8571934915b76ba6/users/5f66c5b0b6dd4c488782a8f1a85fe80b/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:47.349375 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 943/1885] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:47.351002 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db021809-4c80-4b56-b757-16bcc4c9fd54 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:47.352564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db021809-4c80-4b56-b757-16bcc4c9fd54 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:47.352719 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94e643-0d9d-4dae-92e5-e966d7e7736c None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:47.352903 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94e643-0d9d-4dae-92e5-e966d7e7736c None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:47.353152 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db021809-4c80-4b56-b757-16bcc4c9fd54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:47.353297 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94e643-0d9d-4dae-92e5-e966d7e7736c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:47.374040 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-ed94e643-0d9d-4dae-92e5-e966d7e7736c None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:47.375244 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-ed94e643-0d9d-4dae-92e5-e966d7e7736c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:47.376791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-db021809-4c80-4b56-b757-16bcc4c9fd54 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:47.378113 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-db021809-4c80-4b56-b757-16bcc4c9fd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:47.399155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:47.448641 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 944/1886] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:47.448641 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db021809-4c80-4b56-b757-16bcc4c9fd54 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:47.458484 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db021809-4c80-4b56-b757-16bcc4c9fd54 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:47.459554 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 943/1887] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:47.460708 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94e643-0d9d-4dae-92e5-e966d7e7736c None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:47.463572 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04f088-3d84-42de-860d-cd37cc3fb166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:47.464255 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04f088-3d84-42de-860d-cd37cc3fb166 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:47.464543 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04f088-3d84-42de-860d-cd37cc3fb166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:47.464976 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04f088-3d84-42de-860d-cd37cc3fb166 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:47.466031 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04f088-3d84-42de-860d-cd37cc3fb166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:47.468035 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04f088-3d84-42de-860d-cd37cc3fb166 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:47.472169 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94e643-0d9d-4dae-92e5-e966d7e7736c None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:47.478996 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae4b5a-d52c-4f37-b2f5-fc4d2e90e5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:47.479656 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 945/1888] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:47.480533 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04f088-3d84-42de-860d-cd37cc3fb166 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:47.484713 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae4b5a-d52c-4f37-b2f5-fc4d2e90e5bb admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:47.484713 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae4b5a-d52c-4f37-b2f5-fc4d2e90e5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:47.484713 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae4b5a-d52c-4f37-b2f5-fc4d2e90e5bb admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:47.484713 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ae4b5a-d52c-4f37-b2f5-fc4d2e90e5bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:28:47.485259 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ae4b5a-d52c-4f37-b2f5-fc4d2e90e5bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:47.486974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ae4b5a-d52c-4f37-b2f5-fc4d2e90e5bb admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:47.492745 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04f088-3d84-42de-860d-cd37cc3fb166 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:47.498915 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f114536c-3efe-431e-8dbe-6894701cc302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:47.499815 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114536c-3efe-431e-8dbe-6894701cc302 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:47.500110 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114536c-3efe-431e-8dbe-6894701cc302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:47.500388 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114536c-3efe-431e-8dbe-6894701cc302 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:47.501062 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f114536c-3efe-431e-8dbe-6894701cc302 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:28:47.501718 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f114536c-3efe-431e-8dbe-6894701cc302 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:47.503452 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 944/1889] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:28:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:28:47.507009 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43bc9db-d6e9-4710-8673-062b634b8712 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:47.507009 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43bc9db-d6e9-4710-8673-062b634b8712 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:47.507237 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f114536c-3efe-431e-8dbe-6894701cc302 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:47.507339 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43bc9db-d6e9-4710-8673-062b634b8712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:47.524653 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 946/1890] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:28:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:28:47.524653 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f114536c-3efe-431e-8dbe-6894701cc302 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:47.535646 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f114536c-3efe-431e-8dbe-6894701cc302 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:47.536218 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-c43bc9db-d6e9-4710-8673-062b634b8712 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:47.537540 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-c43bc9db-d6e9-4710-8673-062b634b8712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:47.541137 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a04d2c-e9db-4aa8-9bd2-cc891f7d69f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:47.541839 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a04d2c-e9db-4aa8-9bd2-cc891f7d69f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:47.542116 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a04d2c-e9db-4aa8-9bd2-cc891f7d69f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:47.542387 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a04d2c-e9db-4aa8-9bd2-cc891f7d69f5 admin admin] PATH_INFO: `/v3/projects/703cf460339a4aaf8571934915b76ba6/users/5f66c5b0b6dd4c488782a8f1a85fe80b/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:47.545075 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a04d2c-e9db-4aa8-9bd2-cc891f7d69f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=703cf460339a4aaf8571934915b76ba6, user_id=5f66c5b0b6dd4c488782a8f1a85fe80b, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:47.546841 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 945/1891] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:47.547569 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43bc9db-d6e9-4710-8673-062b634b8712 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:47.547823 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a04d2c-e9db-4aa8-9bd2-cc891f7d69f5 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:47.557558 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43bc9db-d6e9-4710-8673-062b634b8712 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:47.561327 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 947/1892] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:47 2022] PUT /identity/v3/projects/703cf460339a4aaf8571934915b76ba6/users/5f66c5b0b6dd4c488782a8f1a85fe80b/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:47.562732 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a04d2c-e9db-4aa8-9bd2-cc891f7d69f5 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:47.580594 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:47.627861 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43749b26-a791-4fa6-8de0-4d24d38e544e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:47.628391 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43749b26-a791-4fa6-8de0-4d24d38e544e admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:47.628673 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43749b26-a791-4fa6-8de0-4d24d38e544e admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:47.628950 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43749b26-a791-4fa6-8de0-4d24d38e544e admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:47.629612 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43749b26-a791-4fa6-8de0-4d24d38e544e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:47.631616 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43749b26-a791-4fa6-8de0-4d24d38e544e admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:47.659923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a04d2c-e9db-4aa8-9bd2-cc891f7d69f5 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:47.672156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84add7d7-f891-4a75-9520-6fe844b1798d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:47.672156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84add7d7-f891-4a75-9520-6fe844b1798d admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:47.672156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84add7d7-f891-4a75-9520-6fe844b1798d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:47.672156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84add7d7-f891-4a75-9520-6fe844b1798d admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:47.673855 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84add7d7-f891-4a75-9520-6fe844b1798d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:47.679227 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84add7d7-f891-4a75-9520-6fe844b1798d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:47.682876 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 946/1893] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:28:47 2022] POST /identity/v3/projects => generated 373 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:47.683878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43749b26-a791-4fa6-8de0-4d24d38e544e admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:47.720773 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 948/1894] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:28:47 2022] POST /identity/v3/projects => generated 375 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:47.722493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eebcf0-c74d-4a99-9645-b33e8f6a105c None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:47.723475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eebcf0-c74d-4a99-9645-b33e8f6a105c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:47.724683 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eebcf0-c74d-4a99-9645-b33e8f6a105c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:47.767045 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-d4eebcf0-c74d-4a99-9645-b33e8f6a105c None None] MFA Rules not processed for user `5f66c5b0b6dd4c488782a8f1a85fe80b`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:47.769165 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d4eebcf0-c74d-4a99-9645-b33e8f6a105c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:47.798133 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43749b26-a791-4fa6-8de0-4d24d38e544e admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:47.848209 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:47.894357 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cac6dbe-781e-424e-9af0-a1f60909ed9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:47.894993 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac6dbe-781e-424e-9af0-a1f60909ed9c admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:47.895273 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac6dbe-781e-424e-9af0-a1f60909ed9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:47.895772 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac6dbe-781e-424e-9af0-a1f60909ed9c admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:47.896430 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cac6dbe-781e-424e-9af0-a1f60909ed9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:28:47.896733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cac6dbe-781e-424e-9af0-a1f60909ed9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:47.898357 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cac6dbe-781e-424e-9af0-a1f60909ed9c admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:47.914602 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 949/1895] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:47 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:47.918997 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eebcf0-c74d-4a99-9645-b33e8f6a105c None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:47.928597 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 947/1896] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:28:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:28:47.929500 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cac6dbe-781e-424e-9af0-a1f60909ed9c admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:47.934822 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eebcf0-c74d-4a99-9645-b33e8f6a105c None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:47.940336 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78cba9d-80c1-4dc6-b062-cff5c40e3fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:47.941004 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78cba9d-80c1-4dc6-b062-cff5c40e3fa6 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:47.941266 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78cba9d-80c1-4dc6-b062-cff5c40e3fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:47.941847 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78cba9d-80c1-4dc6-b062-cff5c40e3fa6 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:47.942657 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78cba9d-80c1-4dc6-b062-cff5c40e3fa6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:47.944672 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cac6dbe-781e-424e-9af0-a1f60909ed9c admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:47.944791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78cba9d-80c1-4dc6-b062-cff5c40e3fa6 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:47.951822 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a41be1-7f11-495d-8c31-efa5b57d4c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:47.953695 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a41be1-7f11-495d-8c31-efa5b57d4c17 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:47.954820 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a41be1-7f11-495d-8c31-efa5b57d4c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:47.955308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a41be1-7f11-495d-8c31-efa5b57d4c17 admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:47.956250 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a41be1-7f11-495d-8c31-efa5b57d4c17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:47.959359 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a41be1-7f11-495d-8c31-efa5b57d4c17 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:47.980012 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 950/1897] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:28:47 2022] POST /identity/v3/users => generated 287 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:47.982051 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78cba9d-80c1-4dc6-b062-cff5c40e3fa6 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:47.985322 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d78cba9d-80c1-4dc6-b062-cff5c40e3fa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:47.994701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 948/1898] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:28:47 2022] POST /identity/v3/users => generated 288 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:47.995810 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a41be1-7f11-495d-8c31-efa5b57d4c17 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:48.010946 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a41be1-7f11-495d-8c31-efa5b57d4c17 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:48.016984 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd9c69-630c-40b1-a08c-86752daf5b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:48.017756 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd9c69-630c-40b1-a08c-86752daf5b56 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:48.018157 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd9c69-630c-40b1-a08c-86752daf5b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:48.019685 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd9c69-630c-40b1-a08c-86752daf5b56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:48.020922 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dd9c69-630c-40b1-a08c-86752daf5b56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:48.023229 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dd9c69-630c-40b1-a08c-86752daf5b56 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:48.076391 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 949/1899] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:28:47 2022] POST /identity/v3/projects => generated 385 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:48.077990 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd9c69-630c-40b1-a08c-86752daf5b56 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:48.105450 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78cba9d-80c1-4dc6-b062-cff5c40e3fa6 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:48.133485 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:48.187072 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd9c69-630c-40b1-a08c-86752daf5b56 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:48.193878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd737666-4898-4fd9-be59-e851d2e9263a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:48.194160 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd737666-4898-4fd9-be59-e851d2e9263a admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:48.194441 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd737666-4898-4fd9-be59-e851d2e9263a admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:48.194832 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd737666-4898-4fd9-be59-e851d2e9263a admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:48.195412 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd737666-4898-4fd9-be59-e851d2e9263a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:48.197109 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd737666-4898-4fd9-be59-e851d2e9263a admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:48.198706 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd81a3-a824-4f4f-b1cf-20d0d4640310 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:48.199922 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd81a3-a824-4f4f-b1cf-20d0d4640310 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:48.200473 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd81a3-a824-4f4f-b1cf-20d0d4640310 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:48.201814 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd81a3-a824-4f4f-b1cf-20d0d4640310 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:48.233583 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 950/1900] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:48.244961 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd737666-4898-4fd9-be59-e851d2e9263a admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:48.261919 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd737666-4898-4fd9-be59-e851d2e9263a admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:48.276720 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2981b592-ef74-4ed8-99ee-069ead8afc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:48.277400 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981b592-ef74-4ed8-99ee-069ead8afc66 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:48.277681 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981b592-ef74-4ed8-99ee-069ead8afc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:48.277959 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981b592-ef74-4ed8-99ee-069ead8afc66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:48.278630 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2981b592-ef74-4ed8-99ee-069ead8afc66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:48.280303 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2981b592-ef74-4ed8-99ee-069ead8afc66 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:48.299035 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 951/1901] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:48.299929 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2981b592-ef74-4ed8-99ee-069ead8afc66 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:48.304173 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd81a3-a824-4f4f-b1cf-20d0d4640310 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:48.306090 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd81a3-a824-4f4f-b1cf-20d0d4640310 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:48.312530 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2981b592-ef74-4ed8-99ee-069ead8afc66 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:48.317865 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ecb85-1925-41d6-a3d8-2ae6d6e25390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:48.319299 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ecb85-1925-41d6-a3d8-2ae6d6e25390 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:48.319299 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ecb85-1925-41d6-a3d8-2ae6d6e25390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:48.319299 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ecb85-1925-41d6-a3d8-2ae6d6e25390 admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:48.319640 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ecb85-1925-41d6-a3d8-2ae6d6e25390 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:48.321748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ecb85-1925-41d6-a3d8-2ae6d6e25390 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:48.323464 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 951/1902] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:47 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:48.324556 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd81a3-a824-4f4f-b1cf-20d0d4640310 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:48.340130 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd81a3-a824-4f4f-b1cf-20d0d4640310 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:48.347159 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e092c-dd90-4864-a113-d625429baf27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:48.347985 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e092c-dd90-4864-a113-d625429baf27 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:48.348339 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e092c-dd90-4864-a113-d625429baf27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:48.348690 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e092c-dd90-4864-a113-d625429baf27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:48.349792 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1e092c-dd90-4864-a113-d625429baf27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:48.351731 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1e092c-dd90-4864-a113-d625429baf27 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:48.355064 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 952/1903] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:28:48 2022] POST /identity/v3/users => generated 293 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:48.355064 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ecb85-1925-41d6-a3d8-2ae6d6e25390 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:48.367826 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ecb85-1925-41d6-a3d8-2ae6d6e25390 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:48.370541 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 952/1904] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:48.371796 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e092c-dd90-4864-a113-d625429baf27 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:48.389813 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65857be-03b8-4db8-ab21-3cb79c836528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:48.390320 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e092c-dd90-4864-a113-d625429baf27 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:48.391803 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65857be-03b8-4db8-ab21-3cb79c836528 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:48.392187 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65857be-03b8-4db8-ab21-3cb79c836528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:48.392572 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65857be-03b8-4db8-ab21-3cb79c836528 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:48.394270 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65857be-03b8-4db8-ab21-3cb79c836528 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:48.396083 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65857be-03b8-4db8-ab21-3cb79c836528 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:48.414918 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc7a58-2067-4751-9270-8c7b0b406108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:48.416172 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc7a58-2067-4751-9270-8c7b0b406108 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:48.416564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc7a58-2067-4751-9270-8c7b0b406108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:48.416945 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc7a58-2067-4751-9270-8c7b0b406108 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:48.417936 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc7a58-2067-4751-9270-8c7b0b406108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:48.421285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc7a58-2067-4751-9270-8c7b0b406108 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:48.433683 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 953/1905] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:48.434568 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65857be-03b8-4db8-ab21-3cb79c836528 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:48.436484 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 953/1906] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:48.437849 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc7a58-2067-4751-9270-8c7b0b406108 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:48.446180 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65857be-03b8-4db8-ab21-3cb79c836528 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:48.453666 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac56cc-3688-4515-923f-99e2770c0a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:48.454384 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac56cc-3688-4515-923f-99e2770c0a6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:48.455193 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac56cc-3688-4515-923f-99e2770c0a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:48.455193 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac56cc-3688-4515-923f-99e2770c0a6e admin admin] PATH_INFO: `/v3/projects/b0b8155602024bf3a21b7f3d522c9663/users/6744c6959678493f8a379594d2d01f95/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:48.467470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc7a58-2067-4751-9270-8c7b0b406108 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:48.473232 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac56cc-3688-4515-923f-99e2770c0a6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0b8155602024bf3a21b7f3d522c9663, user_id=6744c6959678493f8a379594d2d01f95, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:48.477545 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac56cc-3688-4515-923f-99e2770c0a6e admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:48.479002 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd689a2-b1b9-40df-81c1-61796472fc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:48.479626 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd689a2-b1b9-40df-81c1-61796472fc4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:48.480171 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd689a2-b1b9-40df-81c1-61796472fc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:48.480580 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd689a2-b1b9-40df-81c1-61796472fc4e admin admin] PATH_INFO: `/v3/projects/b3ba09adfbc44f268fc06f37938533c7/users/25e8798c64624ca6b0c3a652b6a7681c/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:48.501844 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 954/1907] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:48 2022] PUT /identity/v3/projects/b0b8155602024bf3a21b7f3d522c9663/users/6744c6959678493f8a379594d2d01f95/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:48.504661 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac56cc-3688-4515-923f-99e2770c0a6e admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:48.510245 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd689a2-b1b9-40df-81c1-61796472fc4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3ba09adfbc44f268fc06f37938533c7, user_id=25e8798c64624ca6b0c3a652b6a7681c, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:48.514637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd689a2-b1b9-40df-81c1-61796472fc4e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:48.546624 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 954/1908] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:48 2022] PUT /identity/v3/projects/b3ba09adfbc44f268fc06f37938533c7/users/25e8798c64624ca6b0c3a652b6a7681c/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:48.548823 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:48.550607 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24ba6b-c15f-4c2b-a089-bbaa09e412b0 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:48.551206 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24ba6b-c15f-4c2b-a089-bbaa09e412b0 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:48.551597 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24ba6b-c15f-4c2b-a089-bbaa09e412b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:48.604168 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-9e24ba6b-c15f-4c2b-a089-bbaa09e412b0 None None] MFA Rules not processed for user `6744c6959678493f8a379594d2d01f95`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:48.605602 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9e24ba6b-c15f-4c2b-a089-bbaa09e412b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:48.617646 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac56cc-3688-4515-923f-99e2770c0a6e admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:48.715924 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706d68e-5b3b-43f9-8e8f-9734becbf9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:48.717073 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706d68e-5b3b-43f9-8e8f-9734becbf9b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:48.717490 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706d68e-5b3b-43f9-8e8f-9734becbf9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:48.717765 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706d68e-5b3b-43f9-8e8f-9734becbf9b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:48.718415 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9706d68e-5b3b-43f9-8e8f-9734becbf9b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:48.720639 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9706d68e-5b3b-43f9-8e8f-9734becbf9b4 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:48.737195 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 955/1909] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:48.737195 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778c1c9-a629-4543-8408-7a97b5a40724 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:48.737195 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778c1c9-a629-4543-8408-7a97b5a40724 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:48.737195 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778c1c9-a629-4543-8408-7a97b5a40724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:48.746986 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 955/1910] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:48.748208 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e24ba6b-c15f-4c2b-a089-bbaa09e412b0 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:48.767917 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e24ba6b-c15f-4c2b-a089-bbaa09e412b0 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:48.773315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9610b3ed-526e-4e32-ad7a-d562f8820864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:48.774146 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9610b3ed-526e-4e32-ad7a-d562f8820864 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:48.774432 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9610b3ed-526e-4e32-ad7a-d562f8820864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:48.774745 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9610b3ed-526e-4e32-ad7a-d562f8820864 admin admin] PATH_INFO: `/v3/projects/c7b5c6d47b20444ba8d43bddab3626f9/users/d076cdb9c9b64cf0a9d5c520bf5cd50e/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:48.788938 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:48.797444 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9610b3ed-526e-4e32-ad7a-d562f8820864 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7b5c6d47b20444ba8d43bddab3626f9, user_id=d076cdb9c9b64cf0a9d5c520bf5cd50e, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:48.798936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-a778c1c9-a629-4543-8408-7a97b5a40724 None None] MFA Rules not processed for user `25e8798c64624ca6b0c3a652b6a7681c`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:48.800753 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a778c1c9-a629-4543-8408-7a97b5a40724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:48.802456 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9610b3ed-526e-4e32-ad7a-d562f8820864 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:48.829355 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 956/1911] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:48 2022] PUT /identity/v3/projects/c7b5c6d47b20444ba8d43bddab3626f9/users/d076cdb9c9b64cf0a9d5c520bf5cd50e/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:48.829355 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9610b3ed-526e-4e32-ad7a-d562f8820864 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:48.887399 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 956/1912] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:48.887399 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953aa9dc-94f1-4a88-ae58-b4c18dddf9aa None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:48.887399 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953aa9dc-94f1-4a88-ae58-b4c18dddf9aa None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:48.887399 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953aa9dc-94f1-4a88-ae58-b4c18dddf9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:48.919385 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-953aa9dc-94f1-4a88-ae58-b4c18dddf9aa None None] MFA Rules not processed for user `d076cdb9c9b64cf0a9d5c520bf5cd50e`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:48.920847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-953aa9dc-94f1-4a88-ae58-b4c18dddf9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:48.945361 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9610b3ed-526e-4e32-ad7a-d562f8820864 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:48.958242 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e16fcd-2f2b-4460-851a-8ff4b3d0bfbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:48.959088 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e16fcd-2f2b-4460-851a-8ff4b3d0bfbd admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:48.959427 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e16fcd-2f2b-4460-851a-8ff4b3d0bfbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:48.959982 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e16fcd-2f2b-4460-851a-8ff4b3d0bfbd admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:48.962491 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e16fcd-2f2b-4460-851a-8ff4b3d0bfbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:48.963181 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e16fcd-2f2b-4460-851a-8ff4b3d0bfbd admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:48.983960 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:48.997289 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 957/1913] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:28:48 2022] POST /identity/v3/projects => generated 375 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:48.997289 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e16fcd-2f2b-4460-851a-8ff4b3d0bfbd admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:49.019221 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 957/1914] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:49.020364 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953aa9dc-94f1-4a88-ae58-b4c18dddf9aa None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:49.039334 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953aa9dc-94f1-4a88-ae58-b4c18dddf9aa None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:49.090769 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e16fcd-2f2b-4460-851a-8ff4b3d0bfbd admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:49.098155 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbecc372-5daa-4003-a84d-e101f052c8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:49.099420 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbecc372-5daa-4003-a84d-e101f052c8a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:49.099795 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbecc372-5daa-4003-a84d-e101f052c8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:49.100717 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbecc372-5daa-4003-a84d-e101f052c8a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:49.113270 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbecc372-5daa-4003-a84d-e101f052c8a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:49.113270 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbecc372-5daa-4003-a84d-e101f052c8a6 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:49.124867 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48c516-fd74-4f13-bcef-ca761e4e4671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:49.125784 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48c516-fd74-4f13-bcef-ca761e4e4671 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:49.126059 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48c516-fd74-4f13-bcef-ca761e4e4671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:49.126328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48c516-fd74-4f13-bcef-ca761e4e4671 admin admin] PATH_INFO: `/v3/users/14645a6efb674666a1b9bde5442f6d0f` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:49.127466 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d48c516-fd74-4f13-bcef-ca761e4e4671 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14645a6efb674666a1b9bde5442f6d0f)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:49.129421 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d48c516-fd74-4f13-bcef-ca761e4e4671 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:49.159481 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 958/1915] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:28:48 2022] POST /identity/v3/projects => generated 377 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:49.159481 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbecc372-5daa-4003-a84d-e101f052c8a6 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:49.195409 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-7d48c516-fd74-4f13-bcef-ca761e4e4671 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14645a6efb674666a1b9bde5442f6d0f'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:49.213424 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-7d48c516-fd74-4f13-bcef-ca761e4e4671 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14645a6efb674666a1b9bde5442f6d0f'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:49.223106 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-7d48c516-fd74-4f13-bcef-ca761e4e4671 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14645a6efb674666a1b9bde5442f6d0f'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:49.231509 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-7d48c516-fd74-4f13-bcef-ca761e4e4671 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14645a6efb674666a1b9bde5442f6d0f'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:49.232972 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-7d48c516-fd74-4f13-bcef-ca761e4e4671 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14645a6efb674666a1b9bde5442f6d0f'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:49.244179 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 958/1916] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:28:49 2022] DELETE /identity/v3/users/14645a6efb674666a1b9bde5442f6d0f => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:49.244179 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48c516-fd74-4f13-bcef-ca761e4e4671 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:49.244179 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7d48c516-fd74-4f13-bcef-ca761e4e4671 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:49.266975 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:49.272910 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbecc372-5daa-4003-a84d-e101f052c8a6 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:49.333916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48c516-fd74-4f13-bcef-ca761e4e4671 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:49.337566 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10eff5-f075-45a8-8235-5d492f28861f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:49.338178 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10eff5-f075-45a8-8235-5d492f28861f service cinder] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:49.338456 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10eff5-f075-45a8-8235-5d492f28861f service cinder] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:49.338798 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10eff5-f075-45a8-8235-5d492f28861f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:49.339885 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a76b37c-090c-45aa-9e94-5314635ed96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:49.340508 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76b37c-090c-45aa-9e94-5314635ed96c admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:49.340791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76b37c-090c-45aa-9e94-5314635ed96c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:49.341069 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76b37c-090c-45aa-9e94-5314635ed96c admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:49.342328 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a76b37c-090c-45aa-9e94-5314635ed96c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:49.342817 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-9c10eff5-f075-45a8-8235-5d492f28861f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:49.344539 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a76b37c-090c-45aa-9e94-5314635ed96c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:49.372680 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 959/1917] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:28:49 2022] POST /identity/v3/users => generated 294 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:49.372680 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a76b37c-090c-45aa-9e94-5314635ed96c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:49.373948 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1a76b37c-090c-45aa-9e94-5314635ed96c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:49.374974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:49.387689 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a76b37c-090c-45aa-9e94-5314635ed96c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:49.393934 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d631e6-4e4c-44ee-a701-73a71a11dca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:49.394503 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d631e6-4e4c-44ee-a701-73a71a11dca3 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:49.394784 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d631e6-4e4c-44ee-a701-73a71a11dca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:49.395063 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d631e6-4e4c-44ee-a701-73a71a11dca3 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:49.396110 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d631e6-4e4c-44ee-a701-73a71a11dca3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:49.397671 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d631e6-4e4c-44ee-a701-73a71a11dca3 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:49.423716 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c10eff5-f075-45a8-8235-5d492f28861f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:49.424250 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 960/1918] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:28:49 2022] POST /identity/v3/users => generated 295 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:49.425551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c10eff5-f075-45a8-8235-5d492f28861f service cinder] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:49.425915 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eba1c7-2b5d-4416-bab7-29e25cf1b81e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:49.426281 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eba1c7-2b5d-4416-bab7-29e25cf1b81e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:49.427282 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eba1c7-2b5d-4416-bab7-29e25cf1b81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:49.435079 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 959/1919] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:49.444604 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10eff5-f075-45a8-8235-5d492f28861f service cinder] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:49.450201 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-9c10eff5-f075-45a8-8235-5d492f28861f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:49.450316 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e3eba1c7-2b5d-4416-bab7-29e25cf1b81e None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:49.451552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e3eba1c7-2b5d-4416-bab7-29e25cf1b81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:49.459640 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 961/1920] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:49.460336 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eba1c7-2b5d-4416-bab7-29e25cf1b81e None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:49.468139 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10eff5-f075-45a8-8235-5d492f28861f service cinder] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:49.470782 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eba1c7-2b5d-4416-bab7-29e25cf1b81e None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:49.474046 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18080f75-93c2-4b40-9dc4-a2b2bd5c769b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:49.474666 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18080f75-93c2-4b40-9dc4-a2b2bd5c769b admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:49.474950 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18080f75-93c2-4b40-9dc4-a2b2bd5c769b admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:49.475469 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18080f75-93c2-4b40-9dc4-a2b2bd5c769b admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:49.476231 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28862b42-89bd-4779-88ac-866d7fd57da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:49.476353 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18080f75-93c2-4b40-9dc4-a2b2bd5c769b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:49.476849 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28862b42-89bd-4779-88ac-866d7fd57da8 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:49.477196 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28862b42-89bd-4779-88ac-866d7fd57da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:49.477668 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28862b42-89bd-4779-88ac-866d7fd57da8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:49.478232 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18080f75-93c2-4b40-9dc4-a2b2bd5c769b admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:49.478329 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28862b42-89bd-4779-88ac-866d7fd57da8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:49.480001 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28862b42-89bd-4779-88ac-866d7fd57da8 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:49.494094 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 962/1921] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:49.494248 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 960/1922] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:49.494330 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28862b42-89bd-4779-88ac-866d7fd57da8 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:49.494408 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18080f75-93c2-4b40-9dc4-a2b2bd5c769b admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:49.494408 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-18080f75-93c2-4b40-9dc4-a2b2bd5c769b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:49.495297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-28862b42-89bd-4779-88ac-866d7fd57da8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:49.509279 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28862b42-89bd-4779-88ac-866d7fd57da8 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:49.514554 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28694e42-a6ab-4cc1-b2ba-0c531938775e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:49.515191 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28694e42-a6ab-4cc1-b2ba-0c531938775e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:49.515475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28694e42-a6ab-4cc1-b2ba-0c531938775e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:49.515754 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28694e42-a6ab-4cc1-b2ba-0c531938775e admin admin] PATH_INFO: `/v3/users/d076cdb9c9b64cf0a9d5c520bf5cd50e` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:49.516900 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28694e42-a6ab-4cc1-b2ba-0c531938775e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d076cdb9c9b64cf0a9d5c520bf5cd50e)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:49.518942 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28694e42-a6ab-4cc1-b2ba-0c531938775e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:49.556673 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18080f75-93c2-4b40-9dc4-a2b2bd5c769b admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:49.560869 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a76da1-9333-4113-89d7-049664588df1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:49.562303 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a76da1-9333-4113-89d7-049664588df1 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:49.562586 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a76da1-9333-4113-89d7-049664588df1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:49.562859 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a76da1-9333-4113-89d7-049664588df1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:49.565717 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-28694e42-a6ab-4cc1-b2ba-0c531938775e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd076cdb9c9b64cf0a9d5c520bf5cd50e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:49.572480 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a76da1-9333-4113-89d7-049664588df1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:49.572604 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-28694e42-a6ab-4cc1-b2ba-0c531938775e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd076cdb9c9b64cf0a9d5c520bf5cd50e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:49.574045 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a76da1-9333-4113-89d7-049664588df1 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:49.584637 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 961/1923] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:49.585431 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a76da1-9333-4113-89d7-049664588df1 service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:49.585566 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-28694e42-a6ab-4cc1-b2ba-0c531938775e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd076cdb9c9b64cf0a9d5c520bf5cd50e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:49.594119 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-28694e42-a6ab-4cc1-b2ba-0c531938775e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd076cdb9c9b64cf0a9d5c520bf5cd50e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:49.595296 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-28694e42-a6ab-4cc1-b2ba-0c531938775e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd076cdb9c9b64cf0a9d5c520bf5cd50e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:49.595572 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a76da1-9333-4113-89d7-049664588df1 service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:49.600755 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d2ad7-c1f5-4b0b-bf31-6dc3d432dd25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:49.604075 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 963/1924] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:28:49 2022] DELETE /identity/v3/users/d076cdb9c9b64cf0a9d5c520bf5cd50e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:49.604199 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d2ad7-c1f5-4b0b-bf31-6dc3d432dd25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:49.604199 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d2ad7-c1f5-4b0b-bf31-6dc3d432dd25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:49.604199 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d2ad7-c1f5-4b0b-bf31-6dc3d432dd25 admin admin] PATH_INFO: `/v3/projects/09d1e733a0544e878462e7ee28272f6d/users/184e5b12e1534a10b1360ef76c35592c/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:49.604425 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28694e42-a6ab-4cc1-b2ba-0c531938775e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:49.604425 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-28694e42-a6ab-4cc1-b2ba-0c531938775e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:49.620956 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162d2ad7-c1f5-4b0b-bf31-6dc3d432dd25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09d1e733a0544e878462e7ee28272f6d, user_id=184e5b12e1534a10b1360ef76c35592c, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:49.624551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162d2ad7-c1f5-4b0b-bf31-6dc3d432dd25 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:49.646489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 962/1925] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:49 2022] PUT /identity/v3/projects/09d1e733a0544e878462e7ee28272f6d/users/184e5b12e1534a10b1360ef76c35592c/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:49.646489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa2723-ec0b-4100-bdbc-62df3a10650f None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:49.646489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa2723-ec0b-4100-bdbc-62df3a10650f None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:49.646489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa2723-ec0b-4100-bdbc-62df3a10650f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:49.652639 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=83,keystone:INSERT=7,keystone:DELETE=10 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:49.679279 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-36fa2723-ec0b-4100-bdbc-62df3a10650f None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:49.680627 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-36fa2723-ec0b-4100-bdbc-62df3a10650f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:49.699305 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28694e42-a6ab-4cc1-b2ba-0c531938775e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:49.784826 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d35761c-c4fa-4199-a99d-7e6f263b3854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:49.785904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35761c-c4fa-4199-a99d-7e6f263b3854 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:49.786531 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35761c-c4fa-4199-a99d-7e6f263b3854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:49.787127 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35761c-c4fa-4199-a99d-7e6f263b3854 admin admin] PATH_INFO: `/v3/projects/2f515f9c4e1941cdab63b75c182fd2c7/users/f69bb42471ad45b7b2bd1b476d781882/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:49.817857 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d35761c-c4fa-4199-a99d-7e6f263b3854 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f515f9c4e1941cdab63b75c182fd2c7, user_id=f69bb42471ad45b7b2bd1b476d781882, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:49.821547 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d35761c-c4fa-4199-a99d-7e6f263b3854 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:49.824471 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 963/1926] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:49.826122 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa2723-ec0b-4100-bdbc-62df3a10650f None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:49.830776 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-36fa2723-ec0b-4100-bdbc-62df3a10650f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:49.849735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 964/1927] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:49 2022] PUT /identity/v3/projects/2f515f9c4e1941cdab63b75c182fd2c7/users/f69bb42471ad45b7b2bd1b476d781882/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:49.852217 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa2723-ec0b-4100-bdbc-62df3a10650f None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:49.853434 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d35761c-c4fa-4199-a99d-7e6f263b3854 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:49.858312 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2d35761c-c4fa-4199-a99d-7e6f263b3854 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:49.901073 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:49.943715 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1cb29-f5da-4f12-b2ad-c302f2d922a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:49.944675 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1cb29-f5da-4f12-b2ad-c302f2d922a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:49.945106 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1cb29-f5da-4f12-b2ad-c302f2d922a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:49.945572 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1cb29-f5da-4f12-b2ad-c302f2d922a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:49.946672 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1cb29-f5da-4f12-b2ad-c302f2d922a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:49.948435 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1cb29-f5da-4f12-b2ad-c302f2d922a1 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:49.969007 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 964/1928] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:49.970654 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d35761c-c4fa-4199-a99d-7e6f263b3854 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:49.972541 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1cb29-f5da-4f12-b2ad-c302f2d922a1 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:49.980264 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f0903-b286-4183-9029-1454b2c9e7cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:49.980909 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f0903-b286-4183-9029-1454b2c9e7cb service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:49.981386 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f0903-b286-4183-9029-1454b2c9e7cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:49.981667 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f0903-b286-4183-9029-1454b2c9e7cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:49.998324 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1cb29-f5da-4f12-b2ad-c302f2d922a1 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:49.999611 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8f0903-b286-4183-9029-1454b2c9e7cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:50.004982 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8f0903-b286-4183-9029-1454b2c9e7cb service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:50.008033 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d92622-d993-44bf-844a-8b5f24400639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:50.008208 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d92622-d993-44bf-844a-8b5f24400639 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:50.008492 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d92622-d993-44bf-844a-8b5f24400639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:50.008771 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d92622-d993-44bf-844a-8b5f24400639 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:50.009509 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d92622-d993-44bf-844a-8b5f24400639 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:50.011109 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d92622-d993-44bf-844a-8b5f24400639 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:50.020379 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 965/1929] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:50.020753 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f0903-b286-4183-9029-1454b2c9e7cb service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:50.022829 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 965/1930] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:50.023745 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d92622-d993-44bf-844a-8b5f24400639 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:50.027028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-84d92622-d993-44bf-844a-8b5f24400639 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:50.040077 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f0903-b286-4183-9029-1454b2c9e7cb service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:50.042227 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d92622-d993-44bf-844a-8b5f24400639 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:50.049961 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259908cc-2b06-4601-ac89-1822ebce4ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:50.050785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259908cc-2b06-4601-ac89-1822ebce4ba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:50.051139 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259908cc-2b06-4601-ac89-1822ebce4ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:50.051485 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259908cc-2b06-4601-ac89-1822ebce4ba6 admin admin] PATH_INFO: `/v3/projects/8ee2ee965b4d497da9ca22bab80d9211` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:50.052738 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259908cc-2b06-4601-ac89-1822ebce4ba6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ee2ee965b4d497da9ca22bab80d9211)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:50.056203 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259908cc-2b06-4601-ac89-1822ebce4ba6 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:50.059951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1ef1a-f0e0-46a8-a4cd-270214944b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:50.061203 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1ef1a-f0e0-46a8-a4cd-270214944b97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:50.061532 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1ef1a-f0e0-46a8-a4cd-270214944b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:50.061813 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1ef1a-f0e0-46a8-a4cd-270214944b97 admin admin] PATH_INFO: `/v3/projects/09d1e733a0544e878462e7ee28272f6d/users/184e5b12e1534a10b1360ef76c35592c/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:50.067868 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d1ef1a-f0e0-46a8-a4cd-270214944b97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09d1e733a0544e878462e7ee28272f6d, user_id=184e5b12e1534a10b1360ef76c35592c, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:50.070740 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d1ef1a-f0e0-46a8-a4cd-270214944b97 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:50.097019 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 966/1931] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:50 2022] PUT /identity/v3/projects/09d1e733a0544e878462e7ee28272f6d/users/184e5b12e1534a10b1360ef76c35592c/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:50.097888 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1ef1a-f0e0-46a8-a4cd-270214944b97 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:50.164557 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-259908cc-2b06-4601-ac89-1822ebce4ba6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ee2ee965b4d497da9ca22bab80d9211', 'request_id': 'req-259908cc-2b06-4601-ac89-1822ebce4ba6'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:50.178280 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-259908cc-2b06-4601-ac89-1822ebce4ba6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ee2ee965b4d497da9ca22bab80d9211', 'request_id': 'req-259908cc-2b06-4601-ac89-1822ebce4ba6'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:50.193170 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-259908cc-2b06-4601-ac89-1822ebce4ba6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ee2ee965b4d497da9ca22bab80d9211', 'request_id': 'req-259908cc-2b06-4601-ac89-1822ebce4ba6'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:50.210208 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-259908cc-2b06-4601-ac89-1822ebce4ba6 admin admin] The token cache is being invalidate because project 8ee2ee965b4d497da9ca22bab80d9211 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:28:50.210597 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-259908cc-2b06-4601-ac89-1822ebce4ba6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:50.211476 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-259908cc-2b06-4601-ac89-1822ebce4ba6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:50.216956 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 966/1932] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:28:50 2022] DELETE /identity/v3/projects/8ee2ee965b4d497da9ca22bab80d9211 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:50.216956 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43820643-1d93-436b-bc2b-51d8f27b8250 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:50.216956 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43820643-1d93-436b-bc2b-51d8f27b8250 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:50.216956 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43820643-1d93-436b-bc2b-51d8f27b8250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:50.246669 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:50.255867 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-43820643-1d93-436b-bc2b-51d8f27b8250 None None] MFA Rules not processed for user `184e5b12e1534a10b1360ef76c35592c`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:50.258206 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-43820643-1d93-436b-bc2b-51d8f27b8250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:50.261308 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1ef1a-f0e0-46a8-a4cd-270214944b97 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:50.357052 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1733aa7-987a-4b1e-91fb-708259b482a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:50.358084 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1733aa7-987a-4b1e-91fb-708259b482a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:50.358586 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1733aa7-987a-4b1e-91fb-708259b482a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:50.359044 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1733aa7-987a-4b1e-91fb-708259b482a5 admin admin] PATH_INFO: `/v3/projects/2f515f9c4e1941cdab63b75c182fd2c7/users/f69bb42471ad45b7b2bd1b476d781882/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:50.362433 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1733aa7-987a-4b1e-91fb-708259b482a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f515f9c4e1941cdab63b75c182fd2c7, user_id=f69bb42471ad45b7b2bd1b476d781882, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:50.366662 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1733aa7-987a-4b1e-91fb-708259b482a5 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:50.373621 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 967/1933] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:50 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:50.375798 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43820643-1d93-436b-bc2b-51d8f27b8250 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:50.385606 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-43820643-1d93-436b-bc2b-51d8f27b8250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:50.393692 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 967/1934] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:50 2022] PUT /identity/v3/projects/2f515f9c4e1941cdab63b75c182fd2c7/users/f69bb42471ad45b7b2bd1b476d781882/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:50.394649 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1733aa7-987a-4b1e-91fb-708259b482a5 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:50.397058 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a1733aa7-987a-4b1e-91fb-708259b482a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:50.413527 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43820643-1d93-436b-bc2b-51d8f27b8250 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:50.435658 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:50.498201 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1733aa7-987a-4b1e-91fb-708259b482a5 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:50.502451 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34301e25-f915-4d31-a188-90d16cc9ea8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:50.503929 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34301e25-f915-4d31-a188-90d16cc9ea8d service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:50.504434 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34301e25-f915-4d31-a188-90d16cc9ea8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:50.504915 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34301e25-f915-4d31-a188-90d16cc9ea8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:50.507548 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-34301e25-f915-4d31-a188-90d16cc9ea8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:50.511028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71feab7b-af98-48b7-a776-10c7d952891c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:50.511768 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71feab7b-af98-48b7-a776-10c7d952891c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:50.512153 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71feab7b-af98-48b7-a776-10c7d952891c admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:50.512480 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71feab7b-af98-48b7-a776-10c7d952891c admin admin] PATH_INFO: `/v3/users/5f66c5b0b6dd4c488782a8f1a85fe80b` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:50.514203 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71feab7b-af98-48b7-a776-10c7d952891c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f66c5b0b6dd4c488782a8f1a85fe80b)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:50.516508 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71feab7b-af98-48b7-a776-10c7d952891c admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:50.592448 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-71feab7b-af98-48b7-a776-10c7d952891c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f66c5b0b6dd4c488782a8f1a85fe80b'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:50.608923 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-71feab7b-af98-48b7-a776-10c7d952891c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f66c5b0b6dd4c488782a8f1a85fe80b'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:50.612481 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34301e25-f915-4d31-a188-90d16cc9ea8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:50.614557 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34301e25-f915-4d31-a188-90d16cc9ea8d service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:50.619193 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-71feab7b-af98-48b7-a776-10c7d952891c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f66c5b0b6dd4c488782a8f1a85fe80b'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:50.635458 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-71feab7b-af98-48b7-a776-10c7d952891c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f66c5b0b6dd4c488782a8f1a85fe80b'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:50.636233 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-71feab7b-af98-48b7-a776-10c7d952891c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f66c5b0b6dd4c488782a8f1a85fe80b'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:50.640425 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 968/1935] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:50.642635 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69256268-af51-4ba5-ac8d-fe3d6a530d4c None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:50.644583 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69256268-af51-4ba5-ac8d-fe3d6a530d4c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:50.644877 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69256268-af51-4ba5-ac8d-fe3d6a530d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:50.650006 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 968/1936] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:28:50 2022] DELETE /identity/v3/users/5f66c5b0b6dd4c488782a8f1a85fe80b => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:50.651063 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71feab7b-af98-48b7-a776-10c7d952891c admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:50.659324 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-71feab7b-af98-48b7-a776-10c7d952891c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:50.703705 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-69256268-af51-4ba5-ac8d-fe3d6a530d4c None None] MFA Rules not processed for user `f69bb42471ad45b7b2bd1b476d781882`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:50.705120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-69256268-af51-4ba5-ac8d-fe3d6a530d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:50.754615 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:50.770799 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71feab7b-af98-48b7-a776-10c7d952891c admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:50.776815 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86863d2-6a97-4eb3-a97e-c778dcb3b8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:50.777838 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86863d2-6a97-4eb3-a97e-c778dcb3b8d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:50.778490 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86863d2-6a97-4eb3-a97e-c778dcb3b8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:50.779008 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86863d2-6a97-4eb3-a97e-c778dcb3b8d6 admin admin] PATH_INFO: `/v3/projects/c7b5c6d47b20444ba8d43bddab3626f9` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:50.780378 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86863d2-6a97-4eb3-a97e-c778dcb3b8d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7b5c6d47b20444ba8d43bddab3626f9)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:50.782650 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86863d2-6a97-4eb3-a97e-c778dcb3b8d6 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:50.826803 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 969/1937] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:50 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:50.834282 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69256268-af51-4ba5-ac8d-fe3d6a530d4c None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:50.836036 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-69256268-af51-4ba5-ac8d-fe3d6a530d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:50.853646 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:50.859468 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-b86863d2-6a97-4eb3-a97e-c778dcb3b8d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7b5c6d47b20444ba8d43bddab3626f9', 'request_id': 'req-b86863d2-6a97-4eb3-a97e-c778dcb3b8d6'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:50.870658 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-b86863d2-6a97-4eb3-a97e-c778dcb3b8d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7b5c6d47b20444ba8d43bddab3626f9', 'request_id': 'req-b86863d2-6a97-4eb3-a97e-c778dcb3b8d6'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:50.894257 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-b86863d2-6a97-4eb3-a97e-c778dcb3b8d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7b5c6d47b20444ba8d43bddab3626f9', 'request_id': 'req-b86863d2-6a97-4eb3-a97e-c778dcb3b8d6'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:50.911648 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69256268-af51-4ba5-ac8d-fe3d6a530d4c None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:50.914839 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-b86863d2-6a97-4eb3-a97e-c778dcb3b8d6 admin admin] The token cache is being invalidate because project c7b5c6d47b20444ba8d43bddab3626f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:28:50.915321 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-b86863d2-6a97-4eb3-a97e-c778dcb3b8d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:50.916228 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-b86863d2-6a97-4eb3-a97e-c778dcb3b8d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:50.921664 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 969/1938] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:28:50 2022] DELETE /identity/v3/projects/c7b5c6d47b20444ba8d43bddab3626f9 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:50.923215 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27cdf77-9abb-41fc-9f49-ac884b9c9f99 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:50.923617 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27cdf77-9abb-41fc-9f49-ac884b9c9f99 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:50.923764 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27cdf77-9abb-41fc-9f49-ac884b9c9f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:50.950412 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-c27cdf77-9abb-41fc-9f49-ac884b9c9f99 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:50.952085 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-c27cdf77-9abb-41fc-9f49-ac884b9c9f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:51.016607 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c454287c-8822-46b8-bea1-5eeba30f7acc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:51.017341 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454287c-8822-46b8-bea1-5eeba30f7acc service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:51.017693 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454287c-8822-46b8-bea1-5eeba30f7acc service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:51.018033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454287c-8822-46b8-bea1-5eeba30f7acc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:51.024206 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c454287c-8822-46b8-bea1-5eeba30f7acc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:51.042982 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 970/1939] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:51.046765 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426fbd4-3972-4079-9635-0bfd5770b350 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:51.047272 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426fbd4-3972-4079-9635-0bfd5770b350 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:51.047838 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426fbd4-3972-4079-9635-0bfd5770b350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:51.077174 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-1426fbd4-3972-4079-9635-0bfd5770b350 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:51.079131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-1426fbd4-3972-4079-9635-0bfd5770b350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:51.090151 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 971/1940] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:51.095053 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426fbd4-3972-4079-9635-0bfd5770b350 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:51.118332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426fbd4-3972-4079-9635-0bfd5770b350 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:51.139056 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd06d1-f55e-4ac8-8413-71353331b255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:51.139701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd06d1-f55e-4ac8-8413-71353331b255 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:51.140006 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd06d1-f55e-4ac8-8413-71353331b255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:51.140354 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd06d1-f55e-4ac8-8413-71353331b255 admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:51.141238 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd06d1-f55e-4ac8-8413-71353331b255 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:28:51.143048 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd06d1-f55e-4ac8-8413-71353331b255 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:51.157950 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd06d1-f55e-4ac8-8413-71353331b255 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:51.170733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 972/1941] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:28:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:28:51.170733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd06d1-f55e-4ac8-8413-71353331b255 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:51.171849 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-31dd06d1-f55e-4ac8-8413-71353331b255 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:51.183722 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c454287c-8822-46b8-bea1-5eeba30f7acc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:51.185752 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c454287c-8822-46b8-bea1-5eeba30f7acc service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:51.187128 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd06d1-f55e-4ac8-8413-71353331b255 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:51.192126 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afde91a-23f2-489a-9218-20d721e843ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:51.192964 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afde91a-23f2-489a-9218-20d721e843ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:51.193407 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afde91a-23f2-489a-9218-20d721e843ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:51.193613 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afde91a-23f2-489a-9218-20d721e843ab admin admin] PATH_INFO: `/v3/projects/703cf460339a4aaf8571934915b76ba6` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:51.195015 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afde91a-23f2-489a-9218-20d721e843ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=703cf460339a4aaf8571934915b76ba6)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:51.197203 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afde91a-23f2-489a-9218-20d721e843ab admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:51.228554 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 970/1942] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 20:28:51.230606 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c454287c-8822-46b8-bea1-5eeba30f7acc service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:51.263901 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c454287c-8822-46b8-bea1-5eeba30f7acc service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:51.277161 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-8afde91a-23f2-489a-9218-20d721e843ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '703cf460339a4aaf8571934915b76ba6', 'request_id': 'req-8afde91a-23f2-489a-9218-20d721e843ab'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:51.285181 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-8afde91a-23f2-489a-9218-20d721e843ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '703cf460339a4aaf8571934915b76ba6', 'request_id': 'req-8afde91a-23f2-489a-9218-20d721e843ab'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:51.293355 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-8afde91a-23f2-489a-9218-20d721e843ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '703cf460339a4aaf8571934915b76ba6', 'request_id': 'req-8afde91a-23f2-489a-9218-20d721e843ab'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:51.308947 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-8afde91a-23f2-489a-9218-20d721e843ab admin admin] The token cache is being invalidate because project 703cf460339a4aaf8571934915b76ba6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:28:51.309554 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-8afde91a-23f2-489a-9218-20d721e843ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:51.310799 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-8afde91a-23f2-489a-9218-20d721e843ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:51.312566 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 973/1943] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:28:51 2022] DELETE /identity/v3/projects/703cf460339a4aaf8571934915b76ba6 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:51.317233 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e9181-c820-428f-aa88-ce55758f0704 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:51.317844 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:51.318678 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e9181-c820-428f-aa88-ce55758f0704 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:51.318963 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e9181-c820-428f-aa88-ce55758f0704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:51.337111 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ff838-8300-4f7e-9c3b-8b16a92ababb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:51.338076 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ff838-8300-4f7e-9c3b-8b16a92ababb admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:51.338352 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ff838-8300-4f7e-9c3b-8b16a92ababb admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:51.338623 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ff838-8300-4f7e-9c3b-8b16a92ababb admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:51.339247 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ff838-8300-4f7e-9c3b-8b16a92ababb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:51.341235 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ff838-8300-4f7e-9c3b-8b16a92ababb admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:51.344273 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-ea6e9181-c820-428f-aa88-ce55758f0704 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:51.346413 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-ea6e9181-c820-428f-aa88-ce55758f0704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:51.356322 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 974/1944] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:28:51.365872 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872df4c-3546-4cf9-982b-86a1b8367d63 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:51.366164 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872df4c-3546-4cf9-982b-86a1b8367d63 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:51.366444 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872df4c-3546-4cf9-982b-86a1b8367d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:51.377104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 971/1945] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:28:51 2022] POST /identity/v3/projects => generated 371 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:51.379048 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ff838-8300-4f7e-9c3b-8b16a92ababb admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:51.381439 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-dc5ff838-8300-4f7e-9c3b-8b16a92ababb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:51.387499 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-f872df4c-3546-4cf9-982b-86a1b8367d63 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:51.388778 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-f872df4c-3546-4cf9-982b-86a1b8367d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:51.431855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:51.501019 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ff838-8300-4f7e-9c3b-8b16a92ababb admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:51.507324 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13f6ea1-7de0-4e43-9d4d-13069e40605c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:51.509423 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f6ea1-7de0-4e43-9d4d-13069e40605c admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:51.509423 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f6ea1-7de0-4e43-9d4d-13069e40605c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:51.509423 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f6ea1-7de0-4e43-9d4d-13069e40605c admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:51.509807 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13f6ea1-7de0-4e43-9d4d-13069e40605c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:51.512933 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13f6ea1-7de0-4e43-9d4d-13069e40605c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:51.538495 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 975/1946] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:28:51.539690 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872df4c-3546-4cf9-982b-86a1b8367d63 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:51.551605 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872df4c-3546-4cf9-982b-86a1b8367d63 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:51.558582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a41ed8d-c089-43d9-b269-d8cf17439528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:51.559476 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41ed8d-c089-43d9-b269-d8cf17439528 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:51.560647 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41ed8d-c089-43d9-b269-d8cf17439528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:51.561133 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41ed8d-c089-43d9-b269-d8cf17439528 admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:51.562137 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a41ed8d-c089-43d9-b269-d8cf17439528 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:28:51.562682 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a41ed8d-c089-43d9-b269-d8cf17439528 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:51.565125 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a41ed8d-c089-43d9-b269-d8cf17439528 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:51.573738 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 972/1947] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:28:51 2022] POST /identity/v3/users => generated 286 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:51.574779 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13f6ea1-7de0-4e43-9d4d-13069e40605c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:51.582363 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 976/1948] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:28:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:28:51.587030 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a41ed8d-c089-43d9-b269-d8cf17439528 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:51.610218 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13f6ea1-7de0-4e43-9d4d-13069e40605c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:51.613026 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a41ed8d-c089-43d9-b269-d8cf17439528 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:51.620911 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3401f5a8-b94e-43b2-9623-7407627fbdcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:51.622917 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401f5a8-b94e-43b2-9623-7407627fbdcd admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:51.622917 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401f5a8-b94e-43b2-9623-7407627fbdcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:51.622917 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401f5a8-b94e-43b2-9623-7407627fbdcd admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:51.625954 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3401f5a8-b94e-43b2-9623-7407627fbdcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:51.626337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304cd305-4127-44a2-be85-a05e33efc520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:51.627032 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304cd305-4127-44a2-be85-a05e33efc520 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:51.627387 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304cd305-4127-44a2-be85-a05e33efc520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:51.627747 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304cd305-4127-44a2-be85-a05e33efc520 admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:51.628706 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304cd305-4127-44a2-be85-a05e33efc520 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:51.631878 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3401f5a8-b94e-43b2-9623-7407627fbdcd admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:51.634216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304cd305-4127-44a2-be85-a05e33efc520 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:51.676748 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 973/1949] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:51 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:51.678007 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3401f5a8-b94e-43b2-9623-7407627fbdcd admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:51.694802 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3401f5a8-b94e-43b2-9623-7407627fbdcd admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:51.704005 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 977/1950] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:28:51 2022] POST /identity/v3/projects => generated 367 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:51.707110 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830b8e26-d74c-4e44-8f80-c797c03a02b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:51.708520 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b8e26-d74c-4e44-8f80-c797c03a02b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:51.708520 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b8e26-d74c-4e44-8f80-c797c03a02b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:51.708520 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b8e26-d74c-4e44-8f80-c797c03a02b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:51.708790 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304cd305-4127-44a2-be85-a05e33efc520 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:51.708933 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830b8e26-d74c-4e44-8f80-c797c03a02b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:51.713674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830b8e26-d74c-4e44-8f80-c797c03a02b4 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:51.756978 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 974/1951] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:51 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:51.761608 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830b8e26-d74c-4e44-8f80-c797c03a02b4 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:51.783209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830b8e26-d74c-4e44-8f80-c797c03a02b4 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:51.804383 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304cd305-4127-44a2-be85-a05e33efc520 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:51.809751 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9035cb2e-ad7f-47e5-862c-8cadb1eebc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:51.810413 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035cb2e-ad7f-47e5-862c-8cadb1eebc3c admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:51.810700 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035cb2e-ad7f-47e5-862c-8cadb1eebc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:51.810977 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035cb2e-ad7f-47e5-862c-8cadb1eebc3c admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:51.811615 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9035cb2e-ad7f-47e5-862c-8cadb1eebc3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:51.813705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9035cb2e-ad7f-47e5-862c-8cadb1eebc3c admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:51.847373 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 978/1952] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:28:51 2022] POST /identity/v3/users => generated 284 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:51.848360 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9035cb2e-ad7f-47e5-862c-8cadb1eebc3c admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:51.862682 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9035cb2e-ad7f-47e5-862c-8cadb1eebc3c admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:51.876127 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfeb5b4-d13d-4146-ac3d-cbabef708d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:51.876940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfeb5b4-d13d-4146-ac3d-cbabef708d86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:51.877231 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfeb5b4-d13d-4146-ac3d-cbabef708d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:51.877622 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfeb5b4-d13d-4146-ac3d-cbabef708d86 admin admin] PATH_INFO: `/v3/projects/32c862d7b7b242f699139ac0821867a8/users/892bbd02129b417c987f4d429e21a833/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:51.878747 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a607f-1e0b-40ab-b535-25f811467754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:51.879383 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a607f-1e0b-40ab-b535-25f811467754 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:51.879675 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a607f-1e0b-40ab-b535-25f811467754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:51.879950 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a607f-1e0b-40ab-b535-25f811467754 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:51.880633 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a607f-1e0b-40ab-b535-25f811467754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:51.882344 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a607f-1e0b-40ab-b535-25f811467754 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:51.894645 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 979/1953] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:51 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:51.898278 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a607f-1e0b-40ab-b535-25f811467754 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:51.904266 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfeb5b4-d13d-4146-ac3d-cbabef708d86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32c862d7b7b242f699139ac0821867a8, user_id=892bbd02129b417c987f4d429e21a833, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:51.909658 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfeb5b4-d13d-4146-ac3d-cbabef708d86 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:51.923160 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a607f-1e0b-40ab-b535-25f811467754 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:51.942079 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 975/1954] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:51 2022] PUT /identity/v3/projects/32c862d7b7b242f699139ac0821867a8/users/892bbd02129b417c987f4d429e21a833/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:51.947564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53678ae-15d8-4a2d-9de6-069a8435da57 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:51.947843 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53678ae-15d8-4a2d-9de6-069a8435da57 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:51.948119 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53678ae-15d8-4a2d-9de6-069a8435da57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:51.995023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e53678ae-15d8-4a2d-9de6-069a8435da57 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:51.996918 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e53678ae-15d8-4a2d-9de6-069a8435da57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:52.028472 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:52.049906 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b7c3ec-e3ba-4114-9889-0492d4014bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:52.050536 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7c3ec-e3ba-4114-9889-0492d4014bee admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:52.050826 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7c3ec-e3ba-4114-9889-0492d4014bee admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:52.051113 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7c3ec-e3ba-4114-9889-0492d4014bee admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:52.051817 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b7c3ec-e3ba-4114-9889-0492d4014bee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:52.053882 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b7c3ec-e3ba-4114-9889-0492d4014bee admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:52.071779 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 980/1955] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:51 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:28:52.073551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b7c3ec-e3ba-4114-9889-0492d4014bee admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:52.087621 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b7c3ec-e3ba-4114-9889-0492d4014bee admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:52.093942 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 976/1956] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:52.097649 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09088bce-10a6-4cfd-af13-998bba67c8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:52.098243 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09088bce-10a6-4cfd-af13-998bba67c8da admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:52.098525 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09088bce-10a6-4cfd-af13-998bba67c8da admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:52.098795 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09088bce-10a6-4cfd-af13-998bba67c8da admin admin] PATH_INFO: `/v3/projects/d2c2fab14f2049e08d457adff92b1122/users/df78fb1303d34e68a070ccc78ce7045c/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:52.102701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:52.109558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53678ae-15d8-4a2d-9de6-069a8435da57 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:52.111355 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e53678ae-15d8-4a2d-9de6-069a8435da57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:52.130011 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09088bce-10a6-4cfd-af13-998bba67c8da admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2c2fab14f2049e08d457adff92b1122, user_id=df78fb1303d34e68a070ccc78ce7045c, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:52.134267 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09088bce-10a6-4cfd-af13-998bba67c8da admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:52.155934 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 981/1957] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:52 2022] PUT /identity/v3/projects/d2c2fab14f2049e08d457adff92b1122/users/df78fb1303d34e68a070ccc78ce7045c/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:52.167585 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899878c0-1ada-43a8-9381-b134bd2cdb4d None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:52.167585 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899878c0-1ada-43a8-9381-b134bd2cdb4d None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:52.167585 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899878c0-1ada-43a8-9381-b134bd2cdb4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:52.205479 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-899878c0-1ada-43a8-9381-b134bd2cdb4d None None] MFA Rules not processed for user `df78fb1303d34e68a070ccc78ce7045c`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:52.207241 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-899878c0-1ada-43a8-9381-b134bd2cdb4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:52.212009 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53678ae-15d8-4a2d-9de6-069a8435da57 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:52.284816 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d83b6a9-4f30-4424-9ab6-8a19624669cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:52.287647 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83b6a9-4f30-4424-9ab6-8a19624669cd service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:52.287953 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83b6a9-4f30-4424-9ab6-8a19624669cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:52.288233 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83b6a9-4f30-4424-9ab6-8a19624669cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:52.426500 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 982/1958] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:52.429133 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899878c0-1ada-43a8-9381-b134bd2cdb4d None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:52.429459 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d83b6a9-4f30-4424-9ab6-8a19624669cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:52.430986 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d83b6a9-4f30-4424-9ab6-8a19624669cd service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:52.451691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899878c0-1ada-43a8-9381-b134bd2cdb4d None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:52.455110 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 977/1959] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 20:28:52.468288 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f03660-4075-442a-9a30-dd30b9ec6c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:52.468924 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f03660-4075-442a-9a30-dd30b9ec6c7f admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:52.469201 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f03660-4075-442a-9a30-dd30b9ec6c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:52.469542 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f03660-4075-442a-9a30-dd30b9ec6c7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:52.470186 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f03660-4075-442a-9a30-dd30b9ec6c7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:52.472218 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f03660-4075-442a-9a30-dd30b9ec6c7f admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:52.536252 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d83b6a9-4f30-4424-9ab6-8a19624669cd service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:52.548101 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 983/1960] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:28:52 2022] POST /identity/v3/projects => generated 369 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:52.553309 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d83b6a9-4f30-4424-9ab6-8a19624669cd service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:52.619087 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:52.690044 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6112294-71ca-4ae0-8d76-9eaa029ca442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:52.691093 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6112294-71ca-4ae0-8d76-9eaa029ca442 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:52.691378 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6112294-71ca-4ae0-8d76-9eaa029ca442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:52.691663 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6112294-71ca-4ae0-8d76-9eaa029ca442 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:52.692295 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6112294-71ca-4ae0-8d76-9eaa029ca442 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:52.695515 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6112294-71ca-4ae0-8d76-9eaa029ca442 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:52.745949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 978/1961] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:28:52 2022] POST /identity/v3/users => generated 291 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:52.746690 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f03660-4075-442a-9a30-dd30b9ec6c7f admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:52.765326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f03660-4075-442a-9a30-dd30b9ec6c7f admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:52.774709 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c6e80-1c68-4794-ab6b-edaff067b967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:52.775521 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c6e80-1c68-4794-ab6b-edaff067b967 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:52.775803 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c6e80-1c68-4794-ab6b-edaff067b967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:52.776073 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c6e80-1c68-4794-ab6b-edaff067b967 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:52.776790 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c6e80-1c68-4794-ab6b-edaff067b967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:52.786053 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c6e80-1c68-4794-ab6b-edaff067b967 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:52.803235 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 984/1962] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:52.804197 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6112294-71ca-4ae0-8d76-9eaa029ca442 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:52.818373 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6112294-71ca-4ae0-8d76-9eaa029ca442 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:52.834745 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a3238-d59f-43e1-b73a-de27436e9b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:52.834745 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a3238-d59f-43e1-b73a-de27436e9b98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:52.834745 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a3238-d59f-43e1-b73a-de27436e9b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:52.834745 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a3238-d59f-43e1-b73a-de27436e9b98 admin admin] PATH_INFO: `/v3/projects/10b5862be3394c3e8f62ffc77a56efdb/users/359f857e84fe4977b803949189ecbc9d/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:52.863589 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a3238-d59f-43e1-b73a-de27436e9b98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10b5862be3394c3e8f62ffc77a56efdb, user_id=359f857e84fe4977b803949189ecbc9d, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:52.868889 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a3238-d59f-43e1-b73a-de27436e9b98 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:52.906777 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 979/1963] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:52 2022] PUT /identity/v3/projects/10b5862be3394c3e8f62ffc77a56efdb/users/359f857e84fe4977b803949189ecbc9d/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:28:52.910703 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c6e80-1c68-4794-ab6b-edaff067b967 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:52.927155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c6e80-1c68-4794-ab6b-edaff067b967 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:53.084645 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca09b611-bbeb-4473-9f3e-24c5fc324d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:53.093735 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca09b611-bbeb-4473-9f3e-24c5fc324d39 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:53.094536 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca09b611-bbeb-4473-9f3e-24c5fc324d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:53.095201 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca09b611-bbeb-4473-9f3e-24c5fc324d39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:53.097660 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca09b611-bbeb-4473-9f3e-24c5fc324d39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:53.102003 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca09b611-bbeb-4473-9f3e-24c5fc324d39 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:53.125447 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a3238-d59f-43e1-b73a-de27436e9b98 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:53.130401 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 985/1964] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:53.140304 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a3238-d59f-43e1-b73a-de27436e9b98 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:53.145627 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2c7f2-7aab-43ec-be33-86968a75391a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:53.146371 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2c7f2-7aab-43ec-be33-86968a75391a admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:53.146711 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2c7f2-7aab-43ec-be33-86968a75391a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:53.147645 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2c7f2-7aab-43ec-be33-86968a75391a admin admin] PATH_INFO: `/v3/projects/10b5862be3394c3e8f62ffc77a56efdb/users/359f857e84fe4977b803949189ecbc9d/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:53.149958 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa2c7f2-7aab-43ec-be33-86968a75391a admin admin] RBAC: Authorizing `identity:create_grant(project_id=10b5862be3394c3e8f62ffc77a56efdb, user_id=359f857e84fe4977b803949189ecbc9d, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:53.152749 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa2c7f2-7aab-43ec-be33-86968a75391a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:53.166236 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 980/1965] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:53 2022] PUT /identity/v3/projects/10b5862be3394c3e8f62ffc77a56efdb/users/359f857e84fe4977b803949189ecbc9d/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:53.180113 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50262a0-77b4-4008-acd0-b6158c65ab12 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:53.180454 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50262a0-77b4-4008-acd0-b6158c65ab12 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:53.180787 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50262a0-77b4-4008-acd0-b6158c65ab12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:53.230988 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-a50262a0-77b4-4008-acd0-b6158c65ab12 None None] MFA Rules not processed for user `359f857e84fe4977b803949189ecbc9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:53.232343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a50262a0-77b4-4008-acd0-b6158c65ab12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:53.271462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:53.350027 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 986/1966] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:53 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:53.362093 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2c7f2-7aab-43ec-be33-86968a75391a admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:53.364792 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6fa2c7f2-7aab-43ec-be33-86968a75391a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:53.433013 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2c7f2-7aab-43ec-be33-86968a75391a admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:53.437274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298cf3a-5aaa-47fe-ba79-592810697a70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:53.437896 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298cf3a-5aaa-47fe-ba79-592810697a70 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:53.438279 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298cf3a-5aaa-47fe-ba79-592810697a70 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:53.438552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298cf3a-5aaa-47fe-ba79-592810697a70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:53.447934 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7298cf3a-5aaa-47fe-ba79-592810697a70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:53.449437 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7298cf3a-5aaa-47fe-ba79-592810697a70 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:53.460783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 981/1967] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:53 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:53.678693 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c03cb-6984-4be7-a5b8-7d4c13fa6527 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:53.678977 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c03cb-6984-4be7-a5b8-7d4c13fa6527 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:53.679250 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c03cb-6984-4be7-a5b8-7d4c13fa6527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:53.695859 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298cf3a-5aaa-47fe-ba79-592810697a70 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:53.698302 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7298cf3a-5aaa-47fe-ba79-592810697a70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:53.725936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-3e8c03cb-6984-4be7-a5b8-7d4c13fa6527 None None] MFA Rules not processed for user `892bbd02129b417c987f4d429e21a833`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:53.727781 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-3e8c03cb-6984-4be7-a5b8-7d4c13fa6527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:53.788208 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298cf3a-5aaa-47fe-ba79-592810697a70 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:53.817591 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab91ebb-59ee-4288-bb94-56d4e7da9c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:53.818129 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 987/1968] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:53.818753 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab91ebb-59ee-4288-bb94-56d4e7da9c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:53.819187 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab91ebb-59ee-4288-bb94-56d4e7da9c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:53.819585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab91ebb-59ee-4288-bb94-56d4e7da9c4b admin admin] PATH_INFO: `/v3/users/6744c6959678493f8a379594d2d01f95` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:53.821329 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab91ebb-59ee-4288-bb94-56d4e7da9c4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6744c6959678493f8a379594d2d01f95)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:53.824112 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab91ebb-59ee-4288-bb94-56d4e7da9c4b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:53.833151 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c03cb-6984-4be7-a5b8-7d4c13fa6527 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:53.835239 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-3e8c03cb-6984-4be7-a5b8-7d4c13fa6527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:53.863175 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c03cb-6984-4be7-a5b8-7d4c13fa6527 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:53.875782 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ae9e9-a1b7-487a-9034-66c41146f3de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:53.876921 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ae9e9-a1b7-487a-9034-66c41146f3de service cinder] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:53.877215 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ae9e9-a1b7-487a-9034-66c41146f3de service cinder] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:53.877519 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ae9e9-a1b7-487a-9034-66c41146f3de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:53.880853 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:53.896456 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ae9e9-a1b7-487a-9034-66c41146f3de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:53.902022 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ae9e9-a1b7-487a-9034-66c41146f3de service cinder] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:53.926555 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 988/1969] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:53.934613 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-eab91ebb-59ee-4288-bb94-56d4e7da9c4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6744c6959678493f8a379594d2d01f95'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:53.947962 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-eab91ebb-59ee-4288-bb94-56d4e7da9c4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6744c6959678493f8a379594d2d01f95'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:53.965634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-eab91ebb-59ee-4288-bb94-56d4e7da9c4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6744c6959678493f8a379594d2d01f95'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:53.976575 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-eab91ebb-59ee-4288-bb94-56d4e7da9c4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6744c6959678493f8a379594d2d01f95'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:53.977766 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-eab91ebb-59ee-4288-bb94-56d4e7da9c4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6744c6959678493f8a379594d2d01f95'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:53.985343 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 982/1970] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:28:53 2022] DELETE /identity/v3/users/6744c6959678493f8a379594d2d01f95 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:28:53.989659 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d7271-b007-4db0-8e17-80da787d087e None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:53.989950 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d7271-b007-4db0-8e17-80da787d087e None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:53.990223 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d7271-b007-4db0-8e17-80da787d087e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:54.016301 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-5b6d7271-b007-4db0-8e17-80da787d087e None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:54.017555 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-5b6d7271-b007-4db0-8e17-80da787d087e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:54.144553 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 989/1971] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:54.158194 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab91ebb-59ee-4288-bb94-56d4e7da9c4b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:54.159818 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-eab91ebb-59ee-4288-bb94-56d4e7da9c4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:54.240025 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab91ebb-59ee-4288-bb94-56d4e7da9c4b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:54.244485 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662bbc6-b5fe-4d31-9536-9f8af790e88d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:54.245296 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662bbc6-b5fe-4d31-9536-9f8af790e88d service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:54.245532 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662bbc6-b5fe-4d31-9536-9f8af790e88d service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:54.245826 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662bbc6-b5fe-4d31-9536-9f8af790e88d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:54.258439 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4662bbc6-b5fe-4d31-9536-9f8af790e88d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:54.260068 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4662bbc6-b5fe-4d31-9536-9f8af790e88d service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:54.273657 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 983/1972] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:54.840223 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d7271-b007-4db0-8e17-80da787d087e None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:54.842536 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-5b6d7271-b007-4db0-8e17-80da787d087e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:54.856339 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d7271-b007-4db0-8e17-80da787d087e None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:54.861827 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766aeb13-3be0-4148-ae84-0b17d48eaa03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:54.862530 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766aeb13-3be0-4148-ae84-0b17d48eaa03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:54.863252 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766aeb13-3be0-4148-ae84-0b17d48eaa03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:54.863825 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766aeb13-3be0-4148-ae84-0b17d48eaa03 admin admin] PATH_INFO: `/v3/projects/b0b8155602024bf3a21b7f3d522c9663` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:54.865127 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766aeb13-3be0-4148-ae84-0b17d48eaa03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0b8155602024bf3a21b7f3d522c9663)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:54.876574 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766aeb13-3be0-4148-ae84-0b17d48eaa03 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:54.927548 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:54.940033 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-766aeb13-3be0-4148-ae84-0b17d48eaa03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0b8155602024bf3a21b7f3d522c9663', 'request_id': 'req-766aeb13-3be0-4148-ae84-0b17d48eaa03'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:54.951378 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-766aeb13-3be0-4148-ae84-0b17d48eaa03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0b8155602024bf3a21b7f3d522c9663', 'request_id': 'req-766aeb13-3be0-4148-ae84-0b17d48eaa03'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:54.960700 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-766aeb13-3be0-4148-ae84-0b17d48eaa03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0b8155602024bf3a21b7f3d522c9663', 'request_id': 'req-766aeb13-3be0-4148-ae84-0b17d48eaa03'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:54.978994 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-766aeb13-3be0-4148-ae84-0b17d48eaa03 admin admin] The token cache is being invalidate because project b0b8155602024bf3a21b7f3d522c9663 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:28:54.979405 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-766aeb13-3be0-4148-ae84-0b17d48eaa03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:54.980232 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-766aeb13-3be0-4148-ae84-0b17d48eaa03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:54.982815 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 990/1973] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:28:54 2022] DELETE /identity/v3/projects/b0b8155602024bf3a21b7f3d522c9663 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:28:54.986740 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662bbc6-b5fe-4d31-9536-9f8af790e88d service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:55.001847 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662bbc6-b5fe-4d31-9536-9f8af790e88d service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:55.070820 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1b49df-3329-41b9-b34e-6cf0298dd80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:55.071603 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b49df-3329-41b9-b34e-6cf0298dd80c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:55.071761 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b49df-3329-41b9-b34e-6cf0298dd80c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:55.072022 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b49df-3329-41b9-b34e-6cf0298dd80c admin admin] PATH_INFO: `/v3/users/184e5b12e1534a10b1360ef76c35592c` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:55.073191 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1b49df-3329-41b9-b34e-6cf0298dd80c admin admin] RBAC: Authorizing `identity:delete_user(user_id=184e5b12e1534a10b1360ef76c35592c)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:55.075232 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1b49df-3329-41b9-b34e-6cf0298dd80c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:55.146124 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:55.149131 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-eb1b49df-3329-41b9-b34e-6cf0298dd80c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '184e5b12e1534a10b1360ef76c35592c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:55.160522 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-eb1b49df-3329-41b9-b34e-6cf0298dd80c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '184e5b12e1534a10b1360ef76c35592c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:55.178064 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-eb1b49df-3329-41b9-b34e-6cf0298dd80c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '184e5b12e1534a10b1360ef76c35592c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:55.199616 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-eb1b49df-3329-41b9-b34e-6cf0298dd80c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '184e5b12e1534a10b1360ef76c35592c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:55.201334 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-eb1b49df-3329-41b9-b34e-6cf0298dd80c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '184e5b12e1534a10b1360ef76c35592c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:55.209352 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 984/1974] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:28:54 2022] DELETE /identity/v3/users/184e5b12e1534a10b1360ef76c35592c => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:55.416616 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766aeb13-3be0-4148-ae84-0b17d48eaa03 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:55.418666 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-766aeb13-3be0-4148-ae84-0b17d48eaa03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:55.443100 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766aeb13-3be0-4148-ae84-0b17d48eaa03 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:55.580379 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195911f-070a-407e-a596-12e97293ae11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:55.581050 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195911f-070a-407e-a596-12e97293ae11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:55.581476 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195911f-070a-407e-a596-12e97293ae11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:55.581757 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195911f-070a-407e-a596-12e97293ae11 admin admin] PATH_INFO: `/v3/users/df78fb1303d34e68a070ccc78ce7045c` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:55.583424 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5195911f-070a-407e-a596-12e97293ae11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df78fb1303d34e68a070ccc78ce7045c)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:55.585487 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5195911f-070a-407e-a596-12e97293ae11 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:55.640324 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1b49df-3329-41b9-b34e-6cf0298dd80c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:55.643538 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-eb1b49df-3329-41b9-b34e-6cf0298dd80c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:55.676980 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1b49df-3329-41b9-b34e-6cf0298dd80c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:55.682227 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da713e27-b184-42da-ab12-3752db74d33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:55.683042 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da713e27-b184-42da-ab12-3752db74d33e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:55.683337 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da713e27-b184-42da-ab12-3752db74d33e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:55.683666 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da713e27-b184-42da-ab12-3752db74d33e admin admin] PATH_INFO: `/v3/projects/09d1e733a0544e878462e7ee28272f6d` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:55.685446 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da713e27-b184-42da-ab12-3752db74d33e admin admin] RBAC: Authorizing `identity:delete_project(project_id=09d1e733a0544e878462e7ee28272f6d)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:55.687767 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da713e27-b184-42da-ab12-3752db74d33e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:55.693770 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-5195911f-070a-407e-a596-12e97293ae11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df78fb1303d34e68a070ccc78ce7045c'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:55.710823 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-5195911f-070a-407e-a596-12e97293ae11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df78fb1303d34e68a070ccc78ce7045c'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:55.720177 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-5195911f-070a-407e-a596-12e97293ae11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df78fb1303d34e68a070ccc78ce7045c'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:55.744722 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-da713e27-b184-42da-ab12-3752db74d33e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09d1e733a0544e878462e7ee28272f6d', 'request_id': 'req-da713e27-b184-42da-ab12-3752db74d33e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:55.749734 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-5195911f-070a-407e-a596-12e97293ae11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df78fb1303d34e68a070ccc78ce7045c'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:55.750380 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-5195911f-070a-407e-a596-12e97293ae11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df78fb1303d34e68a070ccc78ce7045c'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:55.751398 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-da713e27-b184-42da-ab12-3752db74d33e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09d1e733a0544e878462e7ee28272f6d', 'request_id': 'req-da713e27-b184-42da-ab12-3752db74d33e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:55.758393 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-da713e27-b184-42da-ab12-3752db74d33e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09d1e733a0544e878462e7ee28272f6d', 'request_id': 'req-da713e27-b184-42da-ab12-3752db74d33e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:55.760251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 991/1975] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:28:55 2022] DELETE /identity/v3/users/df78fb1303d34e68a070ccc78ce7045c => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:55.765081 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268c38a-19b6-401c-8406-8dfd38ddba77 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:55.765369 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268c38a-19b6-401c-8406-8dfd38ddba77 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:55.765703 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268c38a-19b6-401c-8406-8dfd38ddba77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:55.774674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-da713e27-b184-42da-ab12-3752db74d33e admin admin] The token cache is being invalidate because project 09d1e733a0544e878462e7ee28272f6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:28:55.775434 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-da713e27-b184-42da-ab12-3752db74d33e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:55.776161 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-da713e27-b184-42da-ab12-3752db74d33e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:55.779381 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 985/1976] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:28:55 2022] DELETE /identity/v3/projects/09d1e733a0544e878462e7ee28272f6d => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:55.784526 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c51c81-785e-4d6e-8975-a0c3448224a9 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:55.785488 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c51c81-785e-4d6e-8975-a0c3448224a9 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:55.785967 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c51c81-785e-4d6e-8975-a0c3448224a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:55.789004 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-a268c38a-19b6-401c-8406-8dfd38ddba77 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:55.790648 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a268c38a-19b6-401c-8406-8dfd38ddba77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:55.810063 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-72c51c81-785e-4d6e-8975-a0c3448224a9 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:55.811832 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-72c51c81-785e-4d6e-8975-a0c3448224a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:55.837280 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:55.862647 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 992/1977] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:55.874475 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a268c38a-19b6-401c-8406-8dfd38ddba77 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:55.876096 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a268c38a-19b6-401c-8406-8dfd38ddba77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:55.881956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 986/1978] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:55.890123 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e47eae-6b2d-4b3e-a174-310de467c904 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:55.890411 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e47eae-6b2d-4b3e-a174-310de467c904 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:55.890786 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e47eae-6b2d-4b3e-a174-310de467c904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:55.911551 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-01e47eae-6b2d-4b3e-a174-310de467c904 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:55.913416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-01e47eae-6b2d-4b3e-a174-310de467c904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:55.922793 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 987/1979] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:28:55.926700 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e47eae-6b2d-4b3e-a174-310de467c904 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:55.936404 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e47eae-6b2d-4b3e-a174-310de467c904 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:55.938345 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a268c38a-19b6-401c-8406-8dfd38ddba77 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:55.942206 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48351f45-0120-4e09-b2cc-d2c92daa0116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:55.942666 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae44499c-bc16-49bb-8112-8f53955b935f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:55.942928 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48351f45-0120-4e09-b2cc-d2c92daa0116 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:55.943212 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48351f45-0120-4e09-b2cc-d2c92daa0116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:55.943484 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48351f45-0120-4e09-b2cc-d2c92daa0116 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:55.943587 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44499c-bc16-49bb-8112-8f53955b935f service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:55.944317 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44499c-bc16-49bb-8112-8f53955b935f service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:55.944426 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48351f45-0120-4e09-b2cc-d2c92daa0116 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:28:55.944503 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44499c-bc16-49bb-8112-8f53955b935f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:55.944573 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48351f45-0120-4e09-b2cc-d2c92daa0116 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:55.946299 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48351f45-0120-4e09-b2cc-d2c92daa0116 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:55.955275 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae44499c-bc16-49bb-8112-8f53955b935f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:55.956861 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae44499c-bc16-49bb-8112-8f53955b935f service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:55.962377 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 988/1980] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:28:55.967202 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48351f45-0120-4e09-b2cc-d2c92daa0116 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:55.968107 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 993/1981] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:55.979054 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48351f45-0120-4e09-b2cc-d2c92daa0116 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:55.983798 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ab5ff-daef-44cb-b7ff-eb980271ec49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:55.984412 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ab5ff-daef-44cb-b7ff-eb980271ec49 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:55.984695 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ab5ff-daef-44cb-b7ff-eb980271ec49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:55.984952 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ab5ff-daef-44cb-b7ff-eb980271ec49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:55.985741 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ab5ff-daef-44cb-b7ff-eb980271ec49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:55.989520 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ab5ff-daef-44cb-b7ff-eb980271ec49 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:56.024751 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 989/1982] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:28:55 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:28:56.026599 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae44499c-bc16-49bb-8112-8f53955b935f service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:56.038331 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae44499c-bc16-49bb-8112-8f53955b935f service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:56.117645 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb93f005-3f94-4e9b-bc1b-cf99e6499e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:56.118608 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb93f005-3f94-4e9b-bc1b-cf99e6499e7f admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:56.119091 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb93f005-3f94-4e9b-bc1b-cf99e6499e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:56.119565 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb93f005-3f94-4e9b-bc1b-cf99e6499e7f admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:56.120523 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb93f005-3f94-4e9b-bc1b-cf99e6499e7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:56.123738 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb93f005-3f94-4e9b-bc1b-cf99e6499e7f admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:56.149807 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 994/1983] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:28:56 2022] POST /identity/v3/users => generated 288 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:56.154279 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ab5ff-daef-44cb-b7ff-eb980271ec49 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:56.161358 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:56.171374 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ab5ff-daef-44cb-b7ff-eb980271ec49 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:56.178932 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4ff40-2caf-4ddc-9669-0c7fc1253099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:56.179796 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4ff40-2caf-4ddc-9669-0c7fc1253099 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:56.180246 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4ff40-2caf-4ddc-9669-0c7fc1253099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:56.180610 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4ff40-2caf-4ddc-9669-0c7fc1253099 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:56.181474 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4ff40-2caf-4ddc-9669-0c7fc1253099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:56.183456 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4ff40-2caf-4ddc-9669-0c7fc1253099 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:56.199358 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 990/1984] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:56.201449 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb93f005-3f94-4e9b-bc1b-cf99e6499e7f admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:56.212554 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb93f005-3f94-4e9b-bc1b-cf99e6499e7f admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:56.217711 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0612db-6b94-47ab-adc3-ba71cc8fe8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:56.218440 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0612db-6b94-47ab-adc3-ba71cc8fe8d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:56.218783 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0612db-6b94-47ab-adc3-ba71cc8fe8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:56.219156 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0612db-6b94-47ab-adc3-ba71cc8fe8d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:56.219853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0612db-6b94-47ab-adc3-ba71cc8fe8d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:56.221636 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0612db-6b94-47ab-adc3-ba71cc8fe8d7 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:56.240331 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 995/1985] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:56.242227 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4ff40-2caf-4ddc-9669-0c7fc1253099 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:56.255852 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4ff40-2caf-4ddc-9669-0c7fc1253099 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:56.265312 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c1807-9f9f-4251-9259-24fb0ee9c404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:56.266341 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c1807-9f9f-4251-9259-24fb0ee9c404 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:56.266974 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c1807-9f9f-4251-9259-24fb0ee9c404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:56.267523 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c1807-9f9f-4251-9259-24fb0ee9c404 admin admin] PATH_INFO: `/v3/projects/8a05d3088d3d470781adce690f1f135c/users/c6c81cc986cc4c1f8ea578c330f8822b/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:56.277706 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0612db-6b94-47ab-adc3-ba71cc8fe8d7 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:56.280128 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7c0612db-6b94-47ab-adc3-ba71cc8fe8d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:56.295832 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c1807-9f9f-4251-9259-24fb0ee9c404 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a05d3088d3d470781adce690f1f135c, user_id=c6c81cc986cc4c1f8ea578c330f8822b, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:56.301292 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c1807-9f9f-4251-9259-24fb0ee9c404 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:56.328497 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 991/1986] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:56 2022] PUT /identity/v3/projects/8a05d3088d3d470781adce690f1f135c/users/c6c81cc986cc4c1f8ea578c330f8822b/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:56.329744 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c1807-9f9f-4251-9259-24fb0ee9c404 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:56.339880 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-901c1807-9f9f-4251-9259-24fb0ee9c404 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:56.363516 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0612db-6b94-47ab-adc3-ba71cc8fe8d7 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:56.383246 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:56.415870 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c1807-9f9f-4251-9259-24fb0ee9c404 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:56.421067 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4c679-ec28-4282-9bc7-ee12009c3849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:56.421949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4c679-ec28-4282-9bc7-ee12009c3849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:56.422309 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4c679-ec28-4282-9bc7-ee12009c3849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:56.422591 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4c679-ec28-4282-9bc7-ee12009c3849 admin admin] PATH_INFO: `/v3/projects/d2c2fab14f2049e08d457adff92b1122` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:56.422687 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f37de-83a4-4847-aaeb-8b0099c9ecbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:56.423209 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f37de-83a4-4847-aaeb-8b0099c9ecbd service cinder] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:56.423489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f37de-83a4-4847-aaeb-8b0099c9ecbd service cinder] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:56.423761 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f37de-83a4-4847-aaeb-8b0099c9ecbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:56.424355 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec4c679-ec28-4282-9bc7-ee12009c3849 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2c2fab14f2049e08d457adff92b1122)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:56.425591 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-5d1f37de-83a4-4847-aaeb-8b0099c9ecbd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:56.426669 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec4c679-ec28-4282-9bc7-ee12009c3849 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:56.484487 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4ec4c679-ec28-4282-9bc7-ee12009c3849 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2c2fab14f2049e08d457adff92b1122', 'request_id': 'req-4ec4c679-ec28-4282-9bc7-ee12009c3849'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:56.493524 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4ec4c679-ec28-4282-9bc7-ee12009c3849 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2c2fab14f2049e08d457adff92b1122', 'request_id': 'req-4ec4c679-ec28-4282-9bc7-ee12009c3849'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:56.502689 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4ec4c679-ec28-4282-9bc7-ee12009c3849 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2c2fab14f2049e08d457adff92b1122', 'request_id': 'req-4ec4c679-ec28-4282-9bc7-ee12009c3849'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:56.504539 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1f37de-83a4-4847-aaeb-8b0099c9ecbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:56.506832 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1f37de-83a4-4847-aaeb-8b0099c9ecbd service cinder] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:56.525174 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4ec4c679-ec28-4282-9bc7-ee12009c3849 admin admin] The token cache is being invalidate because project d2c2fab14f2049e08d457adff92b1122 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:28:56.525594 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4ec4c679-ec28-4282-9bc7-ee12009c3849 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:56.526278 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4ec4c679-ec28-4282-9bc7-ee12009c3849 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:56.527569 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 996/1987] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:56 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:56.529045 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfdaa0f-3706-42b5-ba88-1434dcf5f81b None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:56.529458 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfdaa0f-3706-42b5-ba88-1434dcf5f81b None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:56.529729 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfdaa0f-3706-42b5-ba88-1434dcf5f81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:56.529903 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 992/1988] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:28:56 2022] DELETE /identity/v3/projects/d2c2fab14f2049e08d457adff92b1122 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:56.533558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4c679-ec28-4282-9bc7-ee12009c3849 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:56.535242 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4ec4c679-ec28-4282-9bc7-ee12009c3849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:56.565126 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-7bfdaa0f-3706-42b5-ba88-1434dcf5f81b None None] MFA Rules not processed for user `c6c81cc986cc4c1f8ea578c330f8822b`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:56.566361 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7bfdaa0f-3706-42b5-ba88-1434dcf5f81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:56.725385 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 997/1989] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:56.727248 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653a3b1-4ad4-40d1-b528-543cf82f0564 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:56.727664 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653a3b1-4ad4-40d1-b528-543cf82f0564 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:56.728064 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653a3b1-4ad4-40d1-b528-543cf82f0564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:56.730676 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:56.738230 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4c679-ec28-4282-9bc7-ee12009c3849 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:56.754657 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd6661-ebc2-4a03-aadf-2b40d54f3e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:56.755324 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd6661-ebc2-4a03-aadf-2b40d54f3e66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:56.755883 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd6661-ebc2-4a03-aadf-2b40d54f3e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:56.756051 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd6661-ebc2-4a03-aadf-2b40d54f3e66 admin admin] PATH_INFO: `/v3/users/359f857e84fe4977b803949189ecbc9d` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:56.757519 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd6661-ebc2-4a03-aadf-2b40d54f3e66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=359f857e84fe4977b803949189ecbc9d)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:56.759590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd6661-ebc2-4a03-aadf-2b40d54f3e66 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:56.764535 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-5653a3b1-4ad4-40d1-b528-543cf82f0564 None None] MFA Rules not processed for user `892bbd02129b417c987f4d429e21a833`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:56.766188 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-5653a3b1-4ad4-40d1-b528-543cf82f0564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:56.829773 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-65cd6661-ebc2-4a03-aadf-2b40d54f3e66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '359f857e84fe4977b803949189ecbc9d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:56.846028 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-65cd6661-ebc2-4a03-aadf-2b40d54f3e66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '359f857e84fe4977b803949189ecbc9d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:56.876476 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 998/1990] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:56.885522 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-65cd6661-ebc2-4a03-aadf-2b40d54f3e66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '359f857e84fe4977b803949189ecbc9d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:56.890492 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653a3b1-4ad4-40d1-b528-543cf82f0564 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:56.899941 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-5653a3b1-4ad4-40d1-b528-543cf82f0564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:56.901956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-65cd6661-ebc2-4a03-aadf-2b40d54f3e66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '359f857e84fe4977b803949189ecbc9d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:56.903927 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-65cd6661-ebc2-4a03-aadf-2b40d54f3e66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '359f857e84fe4977b803949189ecbc9d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:56.933497 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:56.938595 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 993/1991] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:28:56 2022] DELETE /identity/v3/users/359f857e84fe4977b803949189ecbc9d => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:28:56.942469 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653a3b1-4ad4-40d1-b528-543cf82f0564 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:57.063713 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d1093b-a147-4fd4-921f-3e75468cb04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:57.065133 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d1093b-a147-4fd4-921f-3e75468cb04c admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:57.065766 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d1093b-a147-4fd4-921f-3e75468cb04c admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:57.066188 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d1093b-a147-4fd4-921f-3e75468cb04c admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:57.067073 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d1093b-a147-4fd4-921f-3e75468cb04c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:57.069484 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d1093b-a147-4fd4-921f-3e75468cb04c admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:57.148211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 999/1992] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:28:56 2022] POST /identity/v3/projects => generated 375 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:57.152643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd6661-ebc2-4a03-aadf-2b40d54f3e66 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:57.155393 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-65cd6661-ebc2-4a03-aadf-2b40d54f3e66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:57.206023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd6661-ebc2-4a03-aadf-2b40d54f3e66 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:57.335472 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f93760-a60a-4fc7-8afd-7fd313fbb738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:57.336456 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f93760-a60a-4fc7-8afd-7fd313fbb738 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:57.336944 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f93760-a60a-4fc7-8afd-7fd313fbb738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:57.337477 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f93760-a60a-4fc7-8afd-7fd313fbb738 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:57.338356 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f93760-a60a-4fc7-8afd-7fd313fbb738 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:57.340579 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f93760-a60a-4fc7-8afd-7fd313fbb738 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:57.368859 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 994/1993] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:28:57 2022] POST /identity/v3/users => generated 288 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:57.373689 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d1093b-a147-4fd4-921f-3e75468cb04c admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:57.384285 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d1093b-a147-4fd4-921f-3e75468cb04c admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:57.391569 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591a3a6-ad25-430e-9756-1e17683044a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:57.392367 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591a3a6-ad25-430e-9756-1e17683044a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:57.392756 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591a3a6-ad25-430e-9756-1e17683044a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:57.393035 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591a3a6-ad25-430e-9756-1e17683044a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:57.393929 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9591a3a6-ad25-430e-9756-1e17683044a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:57.395754 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9591a3a6-ad25-430e-9756-1e17683044a5 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:57.408755 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1000/1994] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:28:57.413398 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f93760-a60a-4fc7-8afd-7fd313fbb738 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:57.424384 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f93760-a60a-4fc7-8afd-7fd313fbb738 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:57.429044 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0920d4ac-1ccd-49e3-bf37-46daa4945ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:57.429660 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920d4ac-1ccd-49e3-bf37-46daa4945ba8 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:57.429938 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920d4ac-1ccd-49e3-bf37-46daa4945ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:57.430208 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920d4ac-1ccd-49e3-bf37-46daa4945ba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:57.430852 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0920d4ac-1ccd-49e3-bf37-46daa4945ba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:57.432425 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0920d4ac-1ccd-49e3-bf37-46daa4945ba8 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:57.443775 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 995/1995] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:57.448135 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591a3a6-ad25-430e-9756-1e17683044a5 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:57.458768 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591a3a6-ad25-430e-9756-1e17683044a5 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:57.464088 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425fda55-4486-41b8-9551-4eb53ee1114b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:57.464929 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fda55-4486-41b8-9551-4eb53ee1114b admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:57.465213 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fda55-4486-41b8-9551-4eb53ee1114b admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:57.466249 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fda55-4486-41b8-9551-4eb53ee1114b admin admin] PATH_INFO: `/v3/projects/e38a340029cf4d77be66739813702882/users/c62808b6fe824f16bc0759ebb07626df/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:57.498759 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425fda55-4486-41b8-9551-4eb53ee1114b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e38a340029cf4d77be66739813702882, user_id=c62808b6fe824f16bc0759ebb07626df, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:57.501677 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425fda55-4486-41b8-9551-4eb53ee1114b admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:57.534673 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1001/1996] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:57 2022] PUT /identity/v3/projects/e38a340029cf4d77be66739813702882/users/c62808b6fe824f16bc0759ebb07626df/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:57.545785 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ea775-cac3-47ab-bee8-d1b93b9c272e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:57.546064 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ea775-cac3-47ab-bee8-d1b93b9c272e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:57.546328 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ea775-cac3-47ab-bee8-d1b93b9c272e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:57.578108 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-dc8ea775-cac3-47ab-bee8-d1b93b9c272e None None] MFA Rules not processed for user `c62808b6fe824f16bc0759ebb07626df`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:57.579874 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-dc8ea775-cac3-47ab-bee8-d1b93b9c272e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:57.607727 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425fda55-4486-41b8-9551-4eb53ee1114b admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:57.609974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-425fda55-4486-41b8-9551-4eb53ee1114b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:57.643694 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:57.673588 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 996/1997] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:57.676362 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8ea775-cac3-47ab-bee8-d1b93b9c272e None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:57.705047 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425fda55-4486-41b8-9551-4eb53ee1114b admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:57.710240 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcc535b-6d60-464a-ad97-bb09bc611cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:57.710940 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc535b-6d60-464a-ad97-bb09bc611cf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:57.711287 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc535b-6d60-464a-ad97-bb09bc611cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:57.711626 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc535b-6d60-464a-ad97-bb09bc611cf4 admin admin] PATH_INFO: `/v3/projects/10b5862be3394c3e8f62ffc77a56efdb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:57.712884 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc535b-6d60-464a-ad97-bb09bc611cf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10b5862be3394c3e8f62ffc77a56efdb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:57.715013 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc535b-6d60-464a-ad97-bb09bc611cf4 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:57.751315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8ea775-cac3-47ab-bee8-d1b93b9c272e None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:57.757076 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528f95b-d4c9-4625-a4a0-51bc3077c094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:57.758074 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528f95b-d4c9-4625-a4a0-51bc3077c094 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:57.758626 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528f95b-d4c9-4625-a4a0-51bc3077c094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:57.758918 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528f95b-d4c9-4625-a4a0-51bc3077c094 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:57.759559 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6528f95b-d4c9-4625-a4a0-51bc3077c094 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:57.761540 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6528f95b-d4c9-4625-a4a0-51bc3077c094 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:57.771163 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-5dcc535b-6d60-464a-ad97-bb09bc611cf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10b5862be3394c3e8f62ffc77a56efdb', 'request_id': 'req-5dcc535b-6d60-464a-ad97-bb09bc611cf4'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:57.781672 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-5dcc535b-6d60-464a-ad97-bb09bc611cf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10b5862be3394c3e8f62ffc77a56efdb', 'request_id': 'req-5dcc535b-6d60-464a-ad97-bb09bc611cf4'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:57.783166 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:57.789621 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-5dcc535b-6d60-464a-ad97-bb09bc611cf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10b5862be3394c3e8f62ffc77a56efdb', 'request_id': 'req-5dcc535b-6d60-464a-ad97-bb09bc611cf4'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:57.793334 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 997/1998] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:28:57 2022] POST /identity/v3/projects => generated 377 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:57.797528 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528f95b-d4c9-4625-a4a0-51bc3077c094 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:57.809587 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-5dcc535b-6d60-464a-ad97-bb09bc611cf4 admin admin] The token cache is being invalidate because project 10b5862be3394c3e8f62ffc77a56efdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:28:57.810358 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-5dcc535b-6d60-464a-ad97-bb09bc611cf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:57.811091 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-5dcc535b-6d60-464a-ad97-bb09bc611cf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:28:57.815712 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1002/1999] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:28:57 2022] DELETE /identity/v3/projects/10b5862be3394c3e8f62ffc77a56efdb => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:28:57.818443 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f56c6-d0be-493c-a1c1-1bfb67a046ea None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:57.818733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f56c6-d0be-493c-a1c1-1bfb67a046ea None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:57.819047 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f56c6-d0be-493c-a1c1-1bfb67a046ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:57.840161 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-bc2f56c6-d0be-493c-a1c1-1bfb67a046ea None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:57.841970 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-bc2f56c6-d0be-493c-a1c1-1bfb67a046ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:57.874343 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528f95b-d4c9-4625-a4a0-51bc3077c094 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:57.880063 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13823db4-22e1-4d68-864e-8ff85102129d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:57.880799 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13823db4-22e1-4d68-864e-8ff85102129d admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:57.881083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13823db4-22e1-4d68-864e-8ff85102129d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:57.881381 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13823db4-22e1-4d68-864e-8ff85102129d admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:57.882020 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13823db4-22e1-4d68-864e-8ff85102129d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:57.883975 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13823db4-22e1-4d68-864e-8ff85102129d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:57.910910 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 998/2000] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:28:57 2022] POST /identity/v3/users => generated 295 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:28:57.911105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1003/2001] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:57.912413 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13823db4-22e1-4d68-864e-8ff85102129d admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:57.915821 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-13823db4-22e1-4d68-864e-8ff85102129d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:57.916449 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51aae70-70cb-49fa-974c-593351baf30e None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:57.916818 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51aae70-70cb-49fa-974c-593351baf30e None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:57.917126 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51aae70-70cb-49fa-974c-593351baf30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:57.932315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13823db4-22e1-4d68-864e-8ff85102129d admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:57.937743 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78ca457-fb9d-4de4-bc84-b31bf2e107ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:57.938425 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ca457-fb9d-4de4-bc84-b31bf2e107ad admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:57.938776 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ca457-fb9d-4de4-bc84-b31bf2e107ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:57.938915 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-f51aae70-70cb-49fa-974c-593351baf30e None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:57.939181 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ca457-fb9d-4de4-bc84-b31bf2e107ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:57.939986 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78ca457-fb9d-4de4-bc84-b31bf2e107ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:57.940339 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-f51aae70-70cb-49fa-974c-593351baf30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:57.941843 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78ca457-fb9d-4de4-bc84-b31bf2e107ad admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:57.951923 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1004/2002] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:28:57.952857 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51aae70-70cb-49fa-974c-593351baf30e None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:57.956358 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 999/2003] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:28:57.962155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51aae70-70cb-49fa-974c-593351baf30e None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:57.962898 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78ca457-fb9d-4de4-bc84-b31bf2e107ad admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:57.967497 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f587e-fc7e-45ce-b4cf-1b3e2562846b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:57.968098 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f587e-fc7e-45ce-b4cf-1b3e2562846b admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:57.968373 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f587e-fc7e-45ce-b4cf-1b3e2562846b admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:57.968700 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f587e-fc7e-45ce-b4cf-1b3e2562846b admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:57.969317 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875f587e-fc7e-45ce-b4cf-1b3e2562846b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:28:57.969617 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875f587e-fc7e-45ce-b4cf-1b3e2562846b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:57.971204 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875f587e-fc7e-45ce-b4cf-1b3e2562846b admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:57.975099 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78ca457-fb9d-4de4-bc84-b31bf2e107ad admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:57.980349 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfcab6-b7cf-4048-b9cf-ffe5481ed79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:57.980990 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfcab6-b7cf-4048-b9cf-ffe5481ed79b admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:57.981593 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfcab6-b7cf-4048-b9cf-ffe5481ed79b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:57.982488 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfcab6-b7cf-4048-b9cf-ffe5481ed79b admin admin] PATH_INFO: `/v3/projects/d780c7d3692444f6ba1b9813429a1d21/users/9c756cb712bb4310bf2c93abdd1193a8/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:57.985300 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1005/2004] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:28:57.988718 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f587e-fc7e-45ce-b4cf-1b3e2562846b admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:58.002762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f587e-fc7e-45ce-b4cf-1b3e2562846b admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:58.003131 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dfcab6-b7cf-4048-b9cf-ffe5481ed79b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d780c7d3692444f6ba1b9813429a1d21, user_id=9c756cb712bb4310bf2c93abdd1193a8, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:58.005878 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dfcab6-b7cf-4048-b9cf-ffe5481ed79b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:58.007652 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e65c592-e761-4664-8046-3f734b7fe354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:58.008325 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65c592-e761-4664-8046-3f734b7fe354 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:58.008611 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65c592-e761-4664-8046-3f734b7fe354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:58.008884 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65c592-e761-4664-8046-3f734b7fe354 admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:58.009544 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e65c592-e761-4664-8046-3f734b7fe354 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:58.011695 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e65c592-e761-4664-8046-3f734b7fe354 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:58.024242 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1000/2005] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:57 2022] PUT /identity/v3/projects/d780c7d3692444f6ba1b9813429a1d21/users/9c756cb712bb4310bf2c93abdd1193a8/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:58.026334 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfcab6-b7cf-4048-b9cf-ffe5481ed79b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:58.045438 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1006/2006] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:28:57 2022] POST /identity/v3/projects => generated 367 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:28:58.049599 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e65c592-e761-4664-8046-3f734b7fe354 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:58.059752 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:58.115211 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfcab6-b7cf-4048-b9cf-ffe5481ed79b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:58.133478 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e65c592-e761-4664-8046-3f734b7fe354 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:58.139675 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a9da4-d78a-4289-85c0-8842288bcc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:58.140721 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a9da4-d78a-4289-85c0-8842288bcc7e admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:58.141003 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a9da4-d78a-4289-85c0-8842288bcc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:58.141532 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a9da4-d78a-4289-85c0-8842288bcc7e admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:58.142648 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a9da4-d78a-4289-85c0-8842288bcc7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:58.144905 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a9da4-d78a-4289-85c0-8842288bcc7e admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:58.180678 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1007/2007] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:28:58 2022] POST /identity/v3/users => generated 284 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:28:58.185339 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a9da4-d78a-4289-85c0-8842288bcc7e admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:58.195920 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a9da4-d78a-4289-85c0-8842288bcc7e admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:58.202628 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2fa7c-9f96-48ca-98de-4116fb6fc881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:58.203281 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2fa7c-9f96-48ca-98de-4116fb6fc881 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:58.203564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2fa7c-9f96-48ca-98de-4116fb6fc881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:58.203965 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2fa7c-9f96-48ca-98de-4116fb6fc881 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:58.204702 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b2fa7c-9f96-48ca-98de-4116fb6fc881 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:58.206517 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b2fa7c-9f96-48ca-98de-4116fb6fc881 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:58.207088 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe81a3-845d-4142-9f47-049aea7b3783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:58.207972 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe81a3-845d-4142-9f47-049aea7b3783 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:58.208153 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe81a3-845d-4142-9f47-049aea7b3783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:58.208876 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe81a3-845d-4142-9f47-049aea7b3783 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:58.209581 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe81a3-845d-4142-9f47-049aea7b3783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:58.211248 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe81a3-845d-4142-9f47-049aea7b3783 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:58.218579 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1001/2008] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:58.222519 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2fa7c-9f96-48ca-98de-4116fb6fc881 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:58.225627 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1008/2009] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:28:58.229892 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe81a3-845d-4142-9f47-049aea7b3783 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:58.233974 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2fa7c-9f96-48ca-98de-4116fb6fc881 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:58.240335 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e9f17-adee-493d-8e3d-498b73520aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:58.241361 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e9f17-adee-493d-8e3d-498b73520aa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:58.241766 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e9f17-adee-493d-8e3d-498b73520aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:58.242116 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e9f17-adee-493d-8e3d-498b73520aa5 admin admin] PATH_INFO: `/v3/projects/d780c7d3692444f6ba1b9813429a1d21/users/9c756cb712bb4310bf2c93abdd1193a8/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:58.244683 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283e9f17-adee-493d-8e3d-498b73520aa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d780c7d3692444f6ba1b9813429a1d21, user_id=9c756cb712bb4310bf2c93abdd1193a8, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:58.246023 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe81a3-845d-4142-9f47-049aea7b3783 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:58.247596 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283e9f17-adee-493d-8e3d-498b73520aa5 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:58.254278 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28531ace-3d2b-48ef-a899-2af988fdd3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:58.254935 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28531ace-3d2b-48ef-a899-2af988fdd3d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:58.255240 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28531ace-3d2b-48ef-a899-2af988fdd3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:58.255581 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28531ace-3d2b-48ef-a899-2af988fdd3d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:58.256325 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28531ace-3d2b-48ef-a899-2af988fdd3d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:58.257962 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28531ace-3d2b-48ef-a899-2af988fdd3d9 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:58.267326 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1002/2010] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:58 2022] PUT /identity/v3/projects/d780c7d3692444f6ba1b9813429a1d21/users/9c756cb712bb4310bf2c93abdd1193a8/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:58.270150 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1009/2011] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:28:58.276284 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e9f17-adee-493d-8e3d-498b73520aa5 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:58.278067 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b376a-097a-4551-ac5e-88b00f1b6ef4 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:58.278385 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b376a-097a-4551-ac5e-88b00f1b6ef4 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:58.278777 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b376a-097a-4551-ac5e-88b00f1b6ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:58.303768 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:58.320580 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-313b376a-097a-4551-ac5e-88b00f1b6ef4 None None] MFA Rules not processed for user `9c756cb712bb4310bf2c93abdd1193a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:58.321818 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-313b376a-097a-4551-ac5e-88b00f1b6ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:58.354522 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e9f17-adee-493d-8e3d-498b73520aa5 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:58.360929 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e685b915-901c-420b-91fc-8ed71ed1654c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:58.361823 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e685b915-901c-420b-91fc-8ed71ed1654c admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:58.362156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e685b915-901c-420b-91fc-8ed71ed1654c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:58.362375 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e685b915-901c-420b-91fc-8ed71ed1654c admin admin] PATH_INFO: `/v3/projects/f706678ae62c4aec8c9b321dc4c02586/users/df9a83da39b144bb992bf3d937edfd0c/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:58.386881 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e685b915-901c-420b-91fc-8ed71ed1654c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f706678ae62c4aec8c9b321dc4c02586, user_id=df9a83da39b144bb992bf3d937edfd0c, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:58.389678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e685b915-901c-420b-91fc-8ed71ed1654c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:58.405295 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1003/2013] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:58 2022] PUT /identity/v3/projects/f706678ae62c4aec8c9b321dc4c02586/users/df9a83da39b144bb992bf3d937edfd0c/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:28:58.405718 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1010/2013] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:58 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:28:58.420434 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b376a-097a-4551-ac5e-88b00f1b6ef4 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:58.420907 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86381b1-e503-4d45-bc94-22fa60e2e945 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:58.421508 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86381b1-e503-4d45-bc94-22fa60e2e945 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:58.421508 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86381b1-e503-4d45-bc94-22fa60e2e945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:58.422241 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-313b376a-097a-4551-ac5e-88b00f1b6ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:58.456583 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f86381b1-e503-4d45-bc94-22fa60e2e945 None None] MFA Rules not processed for user `df9a83da39b144bb992bf3d937edfd0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:58.458073 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f86381b1-e503-4d45-bc94-22fa60e2e945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:58.485668 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:58.486430 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b376a-097a-4551-ac5e-88b00f1b6ef4 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:58.562244 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba097b9-4143-4a55-ade0-2ca701faab52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:58.563246 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba097b9-4143-4a55-ade0-2ca701faab52 service cinder] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:58.563531 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba097b9-4143-4a55-ade0-2ca701faab52 service cinder] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:58.565675 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba097b9-4143-4a55-ade0-2ca701faab52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:58.566138 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-cba097b9-4143-4a55-ade0-2ca701faab52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:58.590553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1004/2014] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:58.595363 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86381b1-e503-4d45-bc94-22fa60e2e945 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:58.647950 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba097b9-4143-4a55-ade0-2ca701faab52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:58.649425 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba097b9-4143-4a55-ade0-2ca701faab52 service cinder] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:58.667857 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1011/2015] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:58.682211 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86381b1-e503-4d45-bc94-22fa60e2e945 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:58.687415 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37831c52-2e41-49be-869c-04076a1ddaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:58.688175 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37831c52-2e41-49be-869c-04076a1ddaa1 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:58.688463 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37831c52-2e41-49be-869c-04076a1ddaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:58.688737 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37831c52-2e41-49be-869c-04076a1ddaa1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:58.689501 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37831c52-2e41-49be-869c-04076a1ddaa1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:58.691334 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37831c52-2e41-49be-869c-04076a1ddaa1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:58.727883 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1005/2016] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:28:58 2022] POST /identity/v3/projects => generated 369 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:28:58.729142 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba097b9-4143-4a55-ade0-2ca701faab52 service cinder] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:58.753969 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba097b9-4143-4a55-ade0-2ca701faab52 service cinder] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:58.753969 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:58.815854 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76985306-531b-4629-8887-9ce662e2dd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:58.816558 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76985306-531b-4629-8887-9ce662e2dd81 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:58.816897 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76985306-531b-4629-8887-9ce662e2dd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:58.817264 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76985306-531b-4629-8887-9ce662e2dd81 admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:58.818031 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76985306-531b-4629-8887-9ce662e2dd81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:58.819970 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76985306-531b-4629-8887-9ce662e2dd81 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:58.850484 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1012/2017] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:28:58 2022] POST /identity/v3/users => generated 291 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:28:58.855581 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37831c52-2e41-49be-869c-04076a1ddaa1 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:58.878839 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37831c52-2e41-49be-869c-04076a1ddaa1 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:58.887335 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4198027-6379-460a-95be-782217b5afb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:58.888156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4198027-6379-460a-95be-782217b5afb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:58.888538 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4198027-6379-460a-95be-782217b5afb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:58.888892 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4198027-6379-460a-95be-782217b5afb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:58.889680 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4198027-6379-460a-95be-782217b5afb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:58.891753 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4198027-6379-460a-95be-782217b5afb5 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:58.903598 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1006/2018] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:58.907817 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76985306-531b-4629-8887-9ce662e2dd81 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:58.919815 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76985306-531b-4629-8887-9ce662e2dd81 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:58.925308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e628ed3-0c92-4e18-a92e-be0d838da37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:58.926866 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e628ed3-0c92-4e18-a92e-be0d838da37a admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:58.927181 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e628ed3-0c92-4e18-a92e-be0d838da37a admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:58.927626 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e628ed3-0c92-4e18-a92e-be0d838da37a admin admin] PATH_INFO: `/v3/projects/242ae3b66ad044c4bffde0538e8fcc99/users/0c0870afd1dd4651bbafc309e2368cf5/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:58.945790 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e628ed3-0c92-4e18-a92e-be0d838da37a admin admin] RBAC: Authorizing `identity:create_grant(project_id=242ae3b66ad044c4bffde0538e8fcc99, user_id=0c0870afd1dd4651bbafc309e2368cf5, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:58.948672 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e628ed3-0c92-4e18-a92e-be0d838da37a admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:58.963999 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1013/2019] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:58 2022] PUT /identity/v3/projects/242ae3b66ad044c4bffde0538e8fcc99/users/0c0870afd1dd4651bbafc309e2368cf5/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:58.968431 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4198027-6379-460a-95be-782217b5afb5 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:58.980809 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4198027-6379-460a-95be-782217b5afb5 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:59.053320 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50cb0a0-77c2-4bf5-99b4-c54174dfe391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:59.053320 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50cb0a0-77c2-4bf5-99b4-c54174dfe391 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:59.053320 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50cb0a0-77c2-4bf5-99b4-c54174dfe391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:59.053320 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50cb0a0-77c2-4bf5-99b4-c54174dfe391 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:59.053803 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50cb0a0-77c2-4bf5-99b4-c54174dfe391 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:59.053803 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50cb0a0-77c2-4bf5-99b4-c54174dfe391 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:59.069327 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1007/2020] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:28:59.071532 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e628ed3-0c92-4e18-a92e-be0d838da37a admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:59.083518 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e628ed3-0c92-4e18-a92e-be0d838da37a admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:59.091704 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780a5f6-8666-42c5-8389-aa5c01b4beab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:59.092378 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780a5f6-8666-42c5-8389-aa5c01b4beab admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:59.092672 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780a5f6-8666-42c5-8389-aa5c01b4beab admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:59.093016 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780a5f6-8666-42c5-8389-aa5c01b4beab admin admin] PATH_INFO: `/v3/projects/242ae3b66ad044c4bffde0538e8fcc99/users/0c0870afd1dd4651bbafc309e2368cf5/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:59.095787 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a780a5f6-8666-42c5-8389-aa5c01b4beab admin admin] RBAC: Authorizing `identity:create_grant(project_id=242ae3b66ad044c4bffde0538e8fcc99, user_id=0c0870afd1dd4651bbafc309e2368cf5, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:59.098470 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a780a5f6-8666-42c5-8389-aa5c01b4beab admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:59.111248 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1014/2021] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:28:59 2022] PUT /identity/v3/projects/242ae3b66ad044c4bffde0538e8fcc99/users/0c0870afd1dd4651bbafc309e2368cf5/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:28:59.123725 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989ca78-1947-4c90-b3de-eca3be311633 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:59.124243 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989ca78-1947-4c90-b3de-eca3be311633 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:59.124724 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989ca78-1947-4c90-b3de-eca3be311633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:59.160824 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-7989ca78-1947-4c90-b3de-eca3be311633 None None] MFA Rules not processed for user `0c0870afd1dd4651bbafc309e2368cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:28:59.162270 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7989ca78-1947-4c90-b3de-eca3be311633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:59.196727 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:28:59.251272 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1008/2022] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:28:59 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:28:59.263087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780a5f6-8666-42c5-8389-aa5c01b4beab admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:59.337859 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780a5f6-8666-42c5-8389-aa5c01b4beab admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:59.342169 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593b040-523c-4078-ad78-c1a587219a4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:59.342781 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593b040-523c-4078-ad78-c1a587219a4d service cinder] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:59.343058 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593b040-523c-4078-ad78-c1a587219a4d service cinder] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:59.343328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593b040-523c-4078-ad78-c1a587219a4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:59.352400 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593b040-523c-4078-ad78-c1a587219a4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:59.353851 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593b040-523c-4078-ad78-c1a587219a4d service cinder] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:59.363715 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1015/2023] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:59 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:28:59.408096 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989ca78-1947-4c90-b3de-eca3be311633 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:28:59.409518 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7989ca78-1947-4c90-b3de-eca3be311633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:28:59.421770 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989ca78-1947-4c90-b3de-eca3be311633 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:28:59.425957 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d638c2f-35fc-4678-86a7-7b9f725ff043 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:28:59.426603 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d638c2f-35fc-4678-86a7-7b9f725ff043 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:28:59.426882 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d638c2f-35fc-4678-86a7-7b9f725ff043 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:28:59.427157 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d638c2f-35fc-4678-86a7-7b9f725ff043 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:28:59.484668 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d638c2f-35fc-4678-86a7-7b9f725ff043 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:28:59.486102 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d638c2f-35fc-4678-86a7-7b9f725ff043 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:28:59.495513 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1009/2024] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:28:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:29:01.467293 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593b040-523c-4078-ad78-c1a587219a4d service cinder] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:01.485586 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593b040-523c-4078-ad78-c1a587219a4d service cinder] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:01.489924 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a06253-5597-4aff-8ccf-da3e424cc3c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:01.491661 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a06253-5597-4aff-8ccf-da3e424cc3c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:01.501339 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a06253-5597-4aff-8ccf-da3e424cc3c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:01.502264 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a06253-5597-4aff-8ccf-da3e424cc3c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:01.510636 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-52a06253-5597-4aff-8ccf-da3e424cc3c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:01.571828 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a06253-5597-4aff-8ccf-da3e424cc3c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:01.573930 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a06253-5597-4aff-8ccf-da3e424cc3c9 service cinder] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:01.581864 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:01.587453 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1016/2025] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:29:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 20:29:02.360725 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d638c2f-35fc-4678-86a7-7b9f725ff043 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:02.370970 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d638c2f-35fc-4678-86a7-7b9f725ff043 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:02.375324 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d1e6a7-61ba-49df-97e7-4e20c5e92b69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:02.376004 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1e6a7-61ba-49df-97e7-4e20c5e92b69 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:02.376352 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1e6a7-61ba-49df-97e7-4e20c5e92b69 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:02.376686 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1e6a7-61ba-49df-97e7-4e20c5e92b69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:02.450691 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d1e6a7-61ba-49df-97e7-4e20c5e92b69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:02.452382 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d1e6a7-61ba-49df-97e7-4e20c5e92b69 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:02.463387 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1010/2026] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:29:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:29:02.967558 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a06253-5597-4aff-8ccf-da3e424cc3c9 service cinder] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:03.048666 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a06253-5597-4aff-8ccf-da3e424cc3c9 service cinder] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:03.053332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694bb2c4-6f80-48c1-b320-9d5a4db2e4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:03.053971 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694bb2c4-6f80-48c1-b320-9d5a4db2e4fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:03.054253 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694bb2c4-6f80-48c1-b320-9d5a4db2e4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:03.054525 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694bb2c4-6f80-48c1-b320-9d5a4db2e4fa admin admin] PATH_INFO: `/v3/users/df9a83da39b144bb992bf3d937edfd0c` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:03.055638 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694bb2c4-6f80-48c1-b320-9d5a4db2e4fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=df9a83da39b144bb992bf3d937edfd0c)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:03.057603 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694bb2c4-6f80-48c1-b320-9d5a4db2e4fa admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:03.105486 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-694bb2c4-6f80-48c1-b320-9d5a4db2e4fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df9a83da39b144bb992bf3d937edfd0c'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:03.119564 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-694bb2c4-6f80-48c1-b320-9d5a4db2e4fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df9a83da39b144bb992bf3d937edfd0c'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:03.126545 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-694bb2c4-6f80-48c1-b320-9d5a4db2e4fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df9a83da39b144bb992bf3d937edfd0c'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:03.134751 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-694bb2c4-6f80-48c1-b320-9d5a4db2e4fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df9a83da39b144bb992bf3d937edfd0c'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:03.135494 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-694bb2c4-6f80-48c1-b320-9d5a4db2e4fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df9a83da39b144bb992bf3d937edfd0c'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:03.141825 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1017/2027] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:29:02 2022] DELETE /identity/v3/users/df9a83da39b144bb992bf3d937edfd0c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:03.146315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff3640-23ee-4468-8bee-20461f2e7167 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:03.146607 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff3640-23ee-4468-8bee-20461f2e7167 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:03.146888 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff3640-23ee-4468-8bee-20461f2e7167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:03.166390 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-79ff3640-23ee-4468-8bee-20461f2e7167 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:03.167615 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-79ff3640-23ee-4468-8bee-20461f2e7167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:03.216856 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:03.228638 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1011/2028] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:03.239016 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694bb2c4-6f80-48c1-b320-9d5a4db2e4fa admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:03.240774 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-694bb2c4-6f80-48c1-b320-9d5a4db2e4fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:03.305827 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694bb2c4-6f80-48c1-b320-9d5a4db2e4fa admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:03.309455 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7091497e-fbb4-42ea-ad3a-49869606e763 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:03.310093 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091497e-fbb4-42ea-ad3a-49869606e763 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:03.310396 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091497e-fbb4-42ea-ad3a-49869606e763 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:03.310639 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091497e-fbb4-42ea-ad3a-49869606e763 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:03.319654 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7091497e-fbb4-42ea-ad3a-49869606e763 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:03.321077 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7091497e-fbb4-42ea-ad3a-49869606e763 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:03.331561 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1018/2029] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:29:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:29:03.705121 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff3640-23ee-4468-8bee-20461f2e7167 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:03.708583 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-79ff3640-23ee-4468-8bee-20461f2e7167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:03.725217 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff3640-23ee-4468-8bee-20461f2e7167 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:03.731294 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d86d8-c1c1-4fc3-9ba7-17bc966f3df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:03.732091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d86d8-c1c1-4fc3-9ba7-17bc966f3df8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:03.732440 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d86d8-c1c1-4fc3-9ba7-17bc966f3df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:03.732777 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d86d8-c1c1-4fc3-9ba7-17bc966f3df8 admin admin] PATH_INFO: `/v3/projects/f706678ae62c4aec8c9b321dc4c02586` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:03.734017 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d86d8-c1c1-4fc3-9ba7-17bc966f3df8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f706678ae62c4aec8c9b321dc4c02586)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:03.736080 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d86d8-c1c1-4fc3-9ba7-17bc966f3df8 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:03.798354 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-910d86d8-c1c1-4fc3-9ba7-17bc966f3df8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f706678ae62c4aec8c9b321dc4c02586', 'request_id': 'req-910d86d8-c1c1-4fc3-9ba7-17bc966f3df8'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:03.805447 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-910d86d8-c1c1-4fc3-9ba7-17bc966f3df8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f706678ae62c4aec8c9b321dc4c02586', 'request_id': 'req-910d86d8-c1c1-4fc3-9ba7-17bc966f3df8'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:03.813702 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-910d86d8-c1c1-4fc3-9ba7-17bc966f3df8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f706678ae62c4aec8c9b321dc4c02586', 'request_id': 'req-910d86d8-c1c1-4fc3-9ba7-17bc966f3df8'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:03.829875 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-910d86d8-c1c1-4fc3-9ba7-17bc966f3df8 admin admin] The token cache is being invalidate because project f706678ae62c4aec8c9b321dc4c02586 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:29:03.830483 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-910d86d8-c1c1-4fc3-9ba7-17bc966f3df8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:03.831171 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-910d86d8-c1c1-4fc3-9ba7-17bc966f3df8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:03.832867 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1012/2030] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:29:03 2022] DELETE /identity/v3/projects/f706678ae62c4aec8c9b321dc4c02586 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:03.836810 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7091497e-fbb4-42ea-ad3a-49869606e763 service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:03.847573 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7091497e-fbb4-42ea-ad3a-49869606e763 service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:03.869965 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:03.918301 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25893d5d-4214-4507-8a92-459bb6a745b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:03.919534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25893d5d-4214-4507-8a92-459bb6a745b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:03.919810 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25893d5d-4214-4507-8a92-459bb6a745b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:03.920075 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25893d5d-4214-4507-8a92-459bb6a745b0 admin admin] PATH_INFO: `/v3/users/0c0870afd1dd4651bbafc309e2368cf5` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:03.921201 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25893d5d-4214-4507-8a92-459bb6a745b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c0870afd1dd4651bbafc309e2368cf5)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:03.923145 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25893d5d-4214-4507-8a92-459bb6a745b0 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:03.970319 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-25893d5d-4214-4507-8a92-459bb6a745b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c0870afd1dd4651bbafc309e2368cf5'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:03.983656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-25893d5d-4214-4507-8a92-459bb6a745b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c0870afd1dd4651bbafc309e2368cf5'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:03.991453 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-25893d5d-4214-4507-8a92-459bb6a745b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c0870afd1dd4651bbafc309e2368cf5'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:04.002158 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-25893d5d-4214-4507-8a92-459bb6a745b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c0870afd1dd4651bbafc309e2368cf5'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:04.003204 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-25893d5d-4214-4507-8a92-459bb6a745b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c0870afd1dd4651bbafc309e2368cf5'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:04.015451 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1019/2031] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:29:03 2022] DELETE /identity/v3/users/0c0870afd1dd4651bbafc309e2368cf5 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:04.387571 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d86d8-c1c1-4fc3-9ba7-17bc966f3df8 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:04.389647 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-910d86d8-c1c1-4fc3-9ba7-17bc966f3df8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:04.402982 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d86d8-c1c1-4fc3-9ba7-17bc966f3df8 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:04.464833 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fc70b9-67e7-4f63-b996-7585806aa1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:04.465512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc70b9-67e7-4f63-b996-7585806aa1d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:04.465824 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc70b9-67e7-4f63-b996-7585806aa1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:04.466100 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc70b9-67e7-4f63-b996-7585806aa1d7 admin admin] PATH_INFO: `/v3/projects/242ae3b66ad044c4bffde0538e8fcc99` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:04.467215 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fc70b9-67e7-4f63-b996-7585806aa1d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=242ae3b66ad044c4bffde0538e8fcc99)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:04.469096 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fc70b9-67e7-4f63-b996-7585806aa1d7 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:04.519393 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e3fc70b9-67e7-4f63-b996-7585806aa1d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '242ae3b66ad044c4bffde0538e8fcc99', 'request_id': 'req-e3fc70b9-67e7-4f63-b996-7585806aa1d7'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:04.524922 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e3fc70b9-67e7-4f63-b996-7585806aa1d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '242ae3b66ad044c4bffde0538e8fcc99', 'request_id': 'req-e3fc70b9-67e7-4f63-b996-7585806aa1d7'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:04.531419 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e3fc70b9-67e7-4f63-b996-7585806aa1d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '242ae3b66ad044c4bffde0538e8fcc99', 'request_id': 'req-e3fc70b9-67e7-4f63-b996-7585806aa1d7'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:04.545020 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e3fc70b9-67e7-4f63-b996-7585806aa1d7 admin admin] The token cache is being invalidate because project 242ae3b66ad044c4bffde0538e8fcc99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:29:04.545454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e3fc70b9-67e7-4f63-b996-7585806aa1d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:04.546084 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e3fc70b9-67e7-4f63-b996-7585806aa1d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:04.547586 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1013/2032] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:29:04 2022] DELETE /identity/v3/projects/242ae3b66ad044c4bffde0538e8fcc99 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:04.552589 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f9e2c-fff1-4a56-b57c-88228964debb None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:04.552882 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f9e2c-fff1-4a56-b57c-88228964debb None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:04.553171 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f9e2c-fff1-4a56-b57c-88228964debb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:04.573309 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-4b0f9e2c-fff1-4a56-b57c-88228964debb None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:04.574525 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-4b0f9e2c-fff1-4a56-b57c-88228964debb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:04.639112 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1020/2033] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:29:04.647040 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3863fd-2dee-4458-a12c-93e728d6027e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:04.647318 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3863fd-2dee-4458-a12c-93e728d6027e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:04.647589 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3863fd-2dee-4458-a12c-93e728d6027e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:04.679126 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-7c3863fd-2dee-4458-a12c-93e728d6027e None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:04.680363 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7c3863fd-2dee-4458-a12c-93e728d6027e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:04.689170 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1014/2034] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:29:04.693416 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0f9e2c-fff1-4a56-b57c-88228964debb None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:04.704047 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0f9e2c-fff1-4a56-b57c-88228964debb None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:04.708831 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206bcf85-0afc-4049-827b-b5aa9902558e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:04.709484 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206bcf85-0afc-4049-827b-b5aa9902558e admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:04.709790 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206bcf85-0afc-4049-827b-b5aa9902558e admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:04.710153 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206bcf85-0afc-4049-827b-b5aa9902558e admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:04.711179 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206bcf85-0afc-4049-827b-b5aa9902558e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:29:04.711179 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206bcf85-0afc-4049-827b-b5aa9902558e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:04.712672 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206bcf85-0afc-4049-827b-b5aa9902558e admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:04.724722 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1021/2035] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:29:04.729284 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3863fd-2dee-4458-a12c-93e728d6027e None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:04.734441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:04.741761 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3863fd-2dee-4458-a12c-93e728d6027e None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:04.748955 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fccdb7-13b5-4010-b444-11851ebaac79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:04.749506 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fccdb7-13b5-4010-b444-11851ebaac79 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:04.749750 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fccdb7-13b5-4010-b444-11851ebaac79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:04.750021 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fccdb7-13b5-4010-b444-11851ebaac79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:04.750665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fccdb7-13b5-4010-b444-11851ebaac79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:04.752709 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fccdb7-13b5-4010-b444-11851ebaac79 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:04.782153 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1015/2036] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:29:04 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:29:04.800090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206bcf85-0afc-4049-827b-b5aa9902558e admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:04.810906 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206bcf85-0afc-4049-827b-b5aa9902558e admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:04.838039 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:04.879169 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a272884-48d7-49f8-a313-59a545a7675d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:04.879862 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a272884-48d7-49f8-a313-59a545a7675d admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:04.880198 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a272884-48d7-49f8-a313-59a545a7675d admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:04.880395 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a272884-48d7-49f8-a313-59a545a7675d admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:04.881049 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a272884-48d7-49f8-a313-59a545a7675d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:04.883008 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a272884-48d7-49f8-a313-59a545a7675d admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:04.905883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1022/2037] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:29:04 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:29:04.909739 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fccdb7-13b5-4010-b444-11851ebaac79 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:04.920272 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fccdb7-13b5-4010-b444-11851ebaac79 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:04.924924 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a8be9-5082-4e6d-8e18-5a544f4d3395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:04.925572 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a8be9-5082-4e6d-8e18-5a544f4d3395 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:04.925854 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a8be9-5082-4e6d-8e18-5a544f4d3395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:04.926134 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a8be9-5082-4e6d-8e18-5a544f4d3395 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:04.926776 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152a8be9-5082-4e6d-8e18-5a544f4d3395 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:04.928364 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152a8be9-5082-4e6d-8e18-5a544f4d3395 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:04.939752 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1016/2038] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:29:04.943710 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a272884-48d7-49f8-a313-59a545a7675d admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:04.954487 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a272884-48d7-49f8-a313-59a545a7675d admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:04.959231 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ce02d-e58d-4ae2-8b97-d7340b0adecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:04.959843 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ce02d-e58d-4ae2-8b97-d7340b0adecc admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:04.960123 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ce02d-e58d-4ae2-8b97-d7340b0adecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:04.960397 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ce02d-e58d-4ae2-8b97-d7340b0adecc admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:04.961044 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77ce02d-e58d-4ae2-8b97-d7340b0adecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:04.962696 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77ce02d-e58d-4ae2-8b97-d7340b0adecc admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:04.974209 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1023/2039] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:29:04.978072 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a8be9-5082-4e6d-8e18-5a544f4d3395 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:04.988398 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a8be9-5082-4e6d-8e18-5a544f4d3395 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:04.993214 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60792f73-6336-4663-b5b2-3c0b135de5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:04.993889 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60792f73-6336-4663-b5b2-3c0b135de5bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:04.994166 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60792f73-6336-4663-b5b2-3c0b135de5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:04.994440 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60792f73-6336-4663-b5b2-3c0b135de5bd admin admin] PATH_INFO: `/v3/projects/971ae822bfd84eacb22fd82a5626d451/users/001eb149f5134a00be66f5203bc1a287/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:05.009063 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60792f73-6336-4663-b5b2-3c0b135de5bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=971ae822bfd84eacb22fd82a5626d451, user_id=001eb149f5134a00be66f5203bc1a287, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:05.011851 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60792f73-6336-4663-b5b2-3c0b135de5bd admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:05.024879 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1017/2040] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:29:04 2022] PUT /identity/v3/projects/971ae822bfd84eacb22fd82a5626d451/users/001eb149f5134a00be66f5203bc1a287/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:05.036708 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafc777-a96d-466b-8789-23b38b37e4c0 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:05.037003 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafc777-a96d-466b-8789-23b38b37e4c0 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:05.037294 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafc777-a96d-466b-8789-23b38b37e4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:05.064234 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-ccafc777-a96d-466b-8789-23b38b37e4c0 None None] MFA Rules not processed for user `001eb149f5134a00be66f5203bc1a287`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:05.065451 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-ccafc777-a96d-466b-8789-23b38b37e4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:05.133537 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1024/2041] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:05.138914 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60792f73-6336-4663-b5b2-3c0b135de5bd admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:05.215148 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60792f73-6336-4663-b5b2-3c0b135de5bd admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:05.219869 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083964af-8d62-44a2-ab77-cec6a69475cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:05.220496 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083964af-8d62-44a2-ab77-cec6a69475cb admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:05.220758 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083964af-8d62-44a2-ab77-cec6a69475cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:05.221030 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083964af-8d62-44a2-ab77-cec6a69475cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:05.221692 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083964af-8d62-44a2-ab77-cec6a69475cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:05.223565 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083964af-8d62-44a2-ab77-cec6a69475cb admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:05.249918 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1018/2042] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:29:05 2022] POST /identity/v3/projects => generated 387 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:29:05.254033 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccafc777-a96d-466b-8789-23b38b37e4c0 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:05.265413 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccafc777-a96d-466b-8789-23b38b37e4c0 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:05.321842 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64fb71e-8fd1-41e3-b254-3d359d6c65c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:05.322361 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fb71e-8fd1-41e3-b254-3d359d6c65c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:05.322638 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fb71e-8fd1-41e3-b254-3d359d6c65c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:05.322908 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fb71e-8fd1-41e3-b254-3d359d6c65c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:05.323539 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64fb71e-8fd1-41e3-b254-3d359d6c65c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:05.326092 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64fb71e-8fd1-41e3-b254-3d359d6c65c1 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:05.348173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1025/2043] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:29:05 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:29:05.352439 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083964af-8d62-44a2-ab77-cec6a69475cb admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:05.369394 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083964af-8d62-44a2-ab77-cec6a69475cb admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:05.377499 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4dca2d-16a2-4c62-b3fa-10f6a40c85e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:05.378532 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4dca2d-16a2-4c62-b3fa-10f6a40c85e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:05.379021 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4dca2d-16a2-4c62-b3fa-10f6a40c85e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:05.379510 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4dca2d-16a2-4c62-b3fa-10f6a40c85e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:05.380673 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4dca2d-16a2-4c62-b3fa-10f6a40c85e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:05.383695 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4dca2d-16a2-4c62-b3fa-10f6a40c85e3 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:05.398913 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1019/2044] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:29:05.402797 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64fb71e-8fd1-41e3-b254-3d359d6c65c1 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:05.413507 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64fb71e-8fd1-41e3-b254-3d359d6c65c1 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:05.418273 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d71a91e-a0af-4127-95cf-336a0a77eb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:05.418942 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71a91e-a0af-4127-95cf-336a0a77eb36 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:05.419219 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71a91e-a0af-4127-95cf-336a0a77eb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:05.419490 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71a91e-a0af-4127-95cf-336a0a77eb36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:05.420126 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d71a91e-a0af-4127-95cf-336a0a77eb36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:05.421744 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d71a91e-a0af-4127-95cf-336a0a77eb36 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:05.433313 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1026/2045] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:29:05.438138 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4dca2d-16a2-4c62-b3fa-10f6a40c85e3 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:05.448464 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4dca2d-16a2-4c62-b3fa-10f6a40c85e3 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:05.453724 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a961d07-d254-4100-b979-7fb30a530620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:05.454371 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a961d07-d254-4100-b979-7fb30a530620 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:05.454647 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a961d07-d254-4100-b979-7fb30a530620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:05.454919 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a961d07-d254-4100-b979-7fb30a530620 admin admin] PATH_INFO: `/v3/projects/4b504d4ede2245729eea1bb81f25519b/users/e61df4944333486d8cae165c7ae0ed51/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:05.469123 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a961d07-d254-4100-b979-7fb30a530620 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b504d4ede2245729eea1bb81f25519b, user_id=e61df4944333486d8cae165c7ae0ed51, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:05.471832 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a961d07-d254-4100-b979-7fb30a530620 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:05.485544 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1020/2046] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:29:05 2022] PUT /identity/v3/projects/4b504d4ede2245729eea1bb81f25519b/users/e61df4944333486d8cae165c7ae0ed51/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:29:05.496026 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f9ee0-d513-4b77-a053-153c08e95c07 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:05.496314 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f9ee0-d513-4b77-a053-153c08e95c07 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:05.496586 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f9ee0-d513-4b77-a053-153c08e95c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:05.529609 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-a37f9ee0-d513-4b77-a053-153c08e95c07 None None] MFA Rules not processed for user `e61df4944333486d8cae165c7ae0ed51`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:05.530883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a37f9ee0-d513-4b77-a053-153c08e95c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:05.535327 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:05.602009 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1027/2047] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:05.607262 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a961d07-d254-4100-b979-7fb30a530620 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:05.667097 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:05.673033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a961d07-d254-4100-b979-7fb30a530620 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:05.678568 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a972886-f815-4746-a9c9-d7a016dc2a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:05.679171 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a972886-f815-4746-a9c9-d7a016dc2a3c admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:05.679450 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a972886-f815-4746-a9c9-d7a016dc2a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:05.679722 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a972886-f815-4746-a9c9-d7a016dc2a3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:05.680347 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a972886-f815-4746-a9c9-d7a016dc2a3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:05.682264 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a972886-f815-4746-a9c9-d7a016dc2a3c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:05.710172 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1021/2048] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:29:05 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:29:05.714421 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f9ee0-d513-4b77-a053-153c08e95c07 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:05.725821 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f9ee0-d513-4b77-a053-153c08e95c07 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:05.857852 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038c1a1-7fef-49a8-947d-5fc5115c5b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:05.858454 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038c1a1-7fef-49a8-947d-5fc5115c5b50 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:05.859188 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038c1a1-7fef-49a8-947d-5fc5115c5b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:05.859444 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038c1a1-7fef-49a8-947d-5fc5115c5b50 admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:05.860114 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0038c1a1-7fef-49a8-947d-5fc5115c5b50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:05.862457 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0038c1a1-7fef-49a8-947d-5fc5115c5b50 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:05.917437 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1028/2049] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:29:05 2022] POST /identity/v3/users => generated 300 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:29:05.918733 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a972886-f815-4746-a9c9-d7a016dc2a3c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:05.933318 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a972886-f815-4746-a9c9-d7a016dc2a3c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:05.938948 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa36c6-67d8-4d39-94e3-5597849e1022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:05.939981 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa36c6-67d8-4d39-94e3-5597849e1022 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:05.940850 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa36c6-67d8-4d39-94e3-5597849e1022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:05.941501 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa36c6-67d8-4d39-94e3-5597849e1022 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:05.942788 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aa36c6-67d8-4d39-94e3-5597849e1022 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:05.944733 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aa36c6-67d8-4d39-94e3-5597849e1022 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:05.962549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1022/2050] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:29:05.965043 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038c1a1-7fef-49a8-947d-5fc5115c5b50 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:05.993272 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038c1a1-7fef-49a8-947d-5fc5115c5b50 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:06.009821 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fbc034-30ed-4d38-b515-9545698c9fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:06.010769 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbc034-30ed-4d38-b515-9545698c9fbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:06.011060 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbc034-30ed-4d38-b515-9545698c9fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:06.011332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbc034-30ed-4d38-b515-9545698c9fbe admin admin] PATH_INFO: `/v3/projects/f0696d1735334419b9c56698bd8a9ff5/users/0506913789984cef915b0c5f62c906dc/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:06.030054 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fbc034-30ed-4d38-b515-9545698c9fbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0696d1735334419b9c56698bd8a9ff5, user_id=0506913789984cef915b0c5f62c906dc, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:06.033033 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fbc034-30ed-4d38-b515-9545698c9fbe admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:06.076172 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1029/2051] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:29:05 2022] PUT /identity/v3/projects/f0696d1735334419b9c56698bd8a9ff5/users/0506913789984cef915b0c5f62c906dc/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:06.080657 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa36c6-67d8-4d39-94e3-5597849e1022 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:06.093914 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa36c6-67d8-4d39-94e3-5597849e1022 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:06.215135 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba75b32-e08e-486c-94c7-a1c1fc0dc26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:06.216323 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba75b32-e08e-486c-94c7-a1c1fc0dc26b admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:06.216657 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba75b32-e08e-486c-94c7-a1c1fc0dc26b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:06.216937 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba75b32-e08e-486c-94c7-a1c1fc0dc26b admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:06.217615 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba75b32-e08e-486c-94c7-a1c1fc0dc26b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:06.219536 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba75b32-e08e-486c-94c7-a1c1fc0dc26b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:06.237448 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1023/2052] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:29:06.238766 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fbc034-30ed-4d38-b515-9545698c9fbe admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:06.256782 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fbc034-30ed-4d38-b515-9545698c9fbe admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:06.267186 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83a933-97dc-4fd0-91bf-0d3697ada1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:06.267869 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83a933-97dc-4fd0-91bf-0d3697ada1b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:06.268152 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83a933-97dc-4fd0-91bf-0d3697ada1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:06.268431 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83a933-97dc-4fd0-91bf-0d3697ada1b3 admin admin] PATH_INFO: `/v3/projects/f0696d1735334419b9c56698bd8a9ff5/users/0506913789984cef915b0c5f62c906dc/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:06.272524 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb83a933-97dc-4fd0-91bf-0d3697ada1b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0696d1735334419b9c56698bd8a9ff5, user_id=0506913789984cef915b0c5f62c906dc, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:06.276930 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb83a933-97dc-4fd0-91bf-0d3697ada1b3 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:06.304630 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1030/2053] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:29:06 2022] PUT /identity/v3/projects/f0696d1735334419b9c56698bd8a9ff5/users/0506913789984cef915b0c5f62c906dc/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:06.311467 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3c7e0-f170-4cd6-ae1b-dfb96fb35a34 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:06.311939 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3c7e0-f170-4cd6-ae1b-dfb96fb35a34 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:06.312391 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3c7e0-f170-4cd6-ae1b-dfb96fb35a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:06.349171 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-26d3c7e0-f170-4cd6-ae1b-dfb96fb35a34 None None] MFA Rules not processed for user `0506913789984cef915b0c5f62c906dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:06.350431 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-26d3c7e0-f170-4cd6-ae1b-dfb96fb35a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:06.421492 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1024/2054] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:06 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:06.432170 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83a933-97dc-4fd0-91bf-0d3697ada1b3 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:06.433882 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-bb83a933-97dc-4fd0-91bf-0d3697ada1b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:06.493396 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83a933-97dc-4fd0-91bf-0d3697ada1b3 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:06.498610 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71995125-d5c4-41d9-a473-cd9405c80c6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:06.499595 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71995125-d5c4-41d9-a473-cd9405c80c6c service cinder] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:06.500063 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71995125-d5c4-41d9-a473-cd9405c80c6c service cinder] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:06.500522 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71995125-d5c4-41d9-a473-cd9405c80c6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:06.511153 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71995125-d5c4-41d9-a473-cd9405c80c6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:06.512878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71995125-d5c4-41d9-a473-cd9405c80c6c service cinder] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:06.519940 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:06.527329 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1031/2055] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:29:06 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:29:07.311317 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d3c7e0-f170-4cd6-ae1b-dfb96fb35a34 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:07.327147 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d3c7e0-f170-4cd6-ae1b-dfb96fb35a34 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:07.331263 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528eb8c-3b63-41f0-9fef-b1f26f0ceac5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:07.332047 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528eb8c-3b63-41f0-9fef-b1f26f0ceac5 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:07.332532 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528eb8c-3b63-41f0-9fef-b1f26f0ceac5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:07.332978 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528eb8c-3b63-41f0-9fef-b1f26f0ceac5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:07.390266 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:07.405525 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528eb8c-3b63-41f0-9fef-b1f26f0ceac5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:07.407781 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528eb8c-3b63-41f0-9fef-b1f26f0ceac5 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:07.422426 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1025/2056] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:29:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:29:12.437083 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71995125-d5c4-41d9-a473-cd9405c80c6c service cinder] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:12.438644 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-71995125-d5c4-41d9-a473-cd9405c80c6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:12.510586 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71995125-d5c4-41d9-a473-cd9405c80c6c service cinder] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:12.516513 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40748ed4-42dc-43ab-ae69-b8e631a0d69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:12.517365 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40748ed4-42dc-43ab-ae69-b8e631a0d69c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:12.517644 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40748ed4-42dc-43ab-ae69-b8e631a0d69c admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:12.517914 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40748ed4-42dc-43ab-ae69-b8e631a0d69c admin admin] PATH_INFO: `/v3/users/c6c81cc986cc4c1f8ea578c330f8822b` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:12.519771 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40748ed4-42dc-43ab-ae69-b8e631a0d69c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6c81cc986cc4c1f8ea578c330f8822b)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:12.522221 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40748ed4-42dc-43ab-ae69-b8e631a0d69c admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:12.586320 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-40748ed4-42dc-43ab-ae69-b8e631a0d69c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6c81cc986cc4c1f8ea578c330f8822b'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:12.603859 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-40748ed4-42dc-43ab-ae69-b8e631a0d69c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6c81cc986cc4c1f8ea578c330f8822b'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:12.616898 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-40748ed4-42dc-43ab-ae69-b8e631a0d69c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6c81cc986cc4c1f8ea578c330f8822b'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:12.626835 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-40748ed4-42dc-43ab-ae69-b8e631a0d69c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6c81cc986cc4c1f8ea578c330f8822b'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:12.627471 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-40748ed4-42dc-43ab-ae69-b8e631a0d69c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6c81cc986cc4c1f8ea578c330f8822b'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:12.634311 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1032/2057] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:29:12 2022] DELETE /identity/v3/users/c6c81cc986cc4c1f8ea578c330f8822b => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:12.639005 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b95c36-7bb9-4d69-be65-ae201df9a1d3 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:12.640170 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b95c36-7bb9-4d69-be65-ae201df9a1d3 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:12.640530 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b95c36-7bb9-4d69-be65-ae201df9a1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:12.661029 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-d8b95c36-7bb9-4d69-be65-ae201df9a1d3 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:12.662428 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d8b95c36-7bb9-4d69-be65-ae201df9a1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:12.731825 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1026/2058] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:12.741711 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40748ed4-42dc-43ab-ae69-b8e631a0d69c admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:12.743152 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-40748ed4-42dc-43ab-ae69-b8e631a0d69c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:12.801434 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40748ed4-42dc-43ab-ae69-b8e631a0d69c admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:12.805140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90a6221-8012-4990-a6cb-f38c6ef641e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:12.805773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a6221-8012-4990-a6cb-f38c6ef641e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:12.806050 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a6221-8012-4990-a6cb-f38c6ef641e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:12.806317 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a6221-8012-4990-a6cb-f38c6ef641e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:12.816289 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90a6221-8012-4990-a6cb-f38c6ef641e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:12.817760 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90a6221-8012-4990-a6cb-f38c6ef641e1 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:12.828018 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1033/2059] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:29:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:29:13.095339 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b95c36-7bb9-4d69-be65-ae201df9a1d3 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:13.096875 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d8b95c36-7bb9-4d69-be65-ae201df9a1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:13.110856 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b95c36-7bb9-4d69-be65-ae201df9a1d3 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:13.116954 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7e88a-6d46-4403-b07b-0e750f0413d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:13.117732 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7e88a-6d46-4403-b07b-0e750f0413d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:13.118086 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7e88a-6d46-4403-b07b-0e750f0413d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:13.118433 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7e88a-6d46-4403-b07b-0e750f0413d1 admin admin] PATH_INFO: `/v3/projects/8a05d3088d3d470781adce690f1f135c` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:13.119823 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7e88a-6d46-4403-b07b-0e750f0413d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a05d3088d3d470781adce690f1f135c)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:13.122355 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7e88a-6d46-4403-b07b-0e750f0413d1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:13.169639 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-67c7e88a-6d46-4403-b07b-0e750f0413d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a05d3088d3d470781adce690f1f135c', 'request_id': 'req-67c7e88a-6d46-4403-b07b-0e750f0413d1'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:13.175045 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-67c7e88a-6d46-4403-b07b-0e750f0413d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a05d3088d3d470781adce690f1f135c', 'request_id': 'req-67c7e88a-6d46-4403-b07b-0e750f0413d1'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:13.181839 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-67c7e88a-6d46-4403-b07b-0e750f0413d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a05d3088d3d470781adce690f1f135c', 'request_id': 'req-67c7e88a-6d46-4403-b07b-0e750f0413d1'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:13.195466 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-67c7e88a-6d46-4403-b07b-0e750f0413d1 admin admin] The token cache is being invalidate because project 8a05d3088d3d470781adce690f1f135c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:29:13.195752 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-67c7e88a-6d46-4403-b07b-0e750f0413d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:13.196364 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-67c7e88a-6d46-4403-b07b-0e750f0413d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:13.197894 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1027/2060] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:29:13 2022] DELETE /identity/v3/projects/8a05d3088d3d470781adce690f1f135c => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:13.202000 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90a6221-8012-4990-a6cb-f38c6ef641e1 service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:13.211774 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90a6221-8012-4990-a6cb-f38c6ef641e1 service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:13.230505 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:13.272182 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd985892-2469-45ae-8a08-e5d08376082a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:13.272819 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd985892-2469-45ae-8a08-e5d08376082a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:13.273116 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd985892-2469-45ae-8a08-e5d08376082a admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:13.273426 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd985892-2469-45ae-8a08-e5d08376082a admin admin] PATH_INFO: `/v3/users/c62808b6fe824f16bc0759ebb07626df` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:13.274563 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd985892-2469-45ae-8a08-e5d08376082a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c62808b6fe824f16bc0759ebb07626df)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:13.276502 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd985892-2469-45ae-8a08-e5d08376082a admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:13.327435 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-cd985892-2469-45ae-8a08-e5d08376082a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c62808b6fe824f16bc0759ebb07626df'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:13.340755 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-cd985892-2469-45ae-8a08-e5d08376082a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c62808b6fe824f16bc0759ebb07626df'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:13.347674 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-cd985892-2469-45ae-8a08-e5d08376082a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c62808b6fe824f16bc0759ebb07626df'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:13.356148 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-cd985892-2469-45ae-8a08-e5d08376082a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c62808b6fe824f16bc0759ebb07626df'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:13.356759 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-cd985892-2469-45ae-8a08-e5d08376082a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c62808b6fe824f16bc0759ebb07626df'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:13.362658 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1034/2061] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:29:13 2022] DELETE /identity/v3/users/c62808b6fe824f16bc0759ebb07626df => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:13.721750 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7e88a-6d46-4403-b07b-0e750f0413d1 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:13.723294 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-67c7e88a-6d46-4403-b07b-0e750f0413d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:13.738865 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7e88a-6d46-4403-b07b-0e750f0413d1 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:13.804088 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a0496d-2cc1-49dd-81a3-79cc4e34951b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:13.804724 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a0496d-2cc1-49dd-81a3-79cc4e34951b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:13.805010 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a0496d-2cc1-49dd-81a3-79cc4e34951b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:13.805312 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a0496d-2cc1-49dd-81a3-79cc4e34951b admin admin] PATH_INFO: `/v3/projects/e38a340029cf4d77be66739813702882` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:13.806549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a0496d-2cc1-49dd-81a3-79cc4e34951b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e38a340029cf4d77be66739813702882)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:13.808462 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a0496d-2cc1-49dd-81a3-79cc4e34951b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:13.827227 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:13.863773 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-28a0496d-2cc1-49dd-81a3-79cc4e34951b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e38a340029cf4d77be66739813702882', 'request_id': 'req-28a0496d-2cc1-49dd-81a3-79cc4e34951b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:13.870365 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-28a0496d-2cc1-49dd-81a3-79cc4e34951b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e38a340029cf4d77be66739813702882', 'request_id': 'req-28a0496d-2cc1-49dd-81a3-79cc4e34951b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:13.877583 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-28a0496d-2cc1-49dd-81a3-79cc4e34951b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e38a340029cf4d77be66739813702882', 'request_id': 'req-28a0496d-2cc1-49dd-81a3-79cc4e34951b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:13.897569 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd985892-2469-45ae-8a08-e5d08376082a admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:13.899586 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-cd985892-2469-45ae-8a08-e5d08376082a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:13.904846 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-28a0496d-2cc1-49dd-81a3-79cc4e34951b admin admin] The token cache is being invalidate because project e38a340029cf4d77be66739813702882 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:29:13.904846 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-28a0496d-2cc1-49dd-81a3-79cc4e34951b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:13.905186 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-28a0496d-2cc1-49dd-81a3-79cc4e34951b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:13.912078 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1028/2062] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:29:13 2022] DELETE /identity/v3/projects/e38a340029cf4d77be66739813702882 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:13.912979 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a0496d-2cc1-49dd-81a3-79cc4e34951b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:13.914551 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-28a0496d-2cc1-49dd-81a3-79cc4e34951b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:14.017221 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a0496d-2cc1-49dd-81a3-79cc4e34951b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:14.023660 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec18fe-f9ca-4b2b-bfe2-071e2fe9b2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:14.024295 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec18fe-f9ca-4b2b-bfe2-071e2fe9b2f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:14.024580 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec18fe-f9ca-4b2b-bfe2-071e2fe9b2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:14.024854 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec18fe-f9ca-4b2b-bfe2-071e2fe9b2f9 admin admin] PATH_INFO: `/v3/users/9c756cb712bb4310bf2c93abdd1193a8` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:14.026059 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec18fe-f9ca-4b2b-bfe2-071e2fe9b2f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c756cb712bb4310bf2c93abdd1193a8)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:14.028050 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec18fe-f9ca-4b2b-bfe2-071e2fe9b2f9 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:14.039843 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd985892-2469-45ae-8a08-e5d08376082a admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:14.089943 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-53ec18fe-f9ca-4b2b-bfe2-071e2fe9b2f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c756cb712bb4310bf2c93abdd1193a8'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:14.097883 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-53ec18fe-f9ca-4b2b-bfe2-071e2fe9b2f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c756cb712bb4310bf2c93abdd1193a8'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:14.110376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd67e51-9766-4ab3-831e-cb619b1c1528 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:14.110991 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-53ec18fe-f9ca-4b2b-bfe2-071e2fe9b2f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c756cb712bb4310bf2c93abdd1193a8'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:14.111129 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd67e51-9766-4ab3-831e-cb619b1c1528 service cinder] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:14.111467 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd67e51-9766-4ab3-831e-cb619b1c1528 service cinder] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:14.111806 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd67e51-9766-4ab3-831e-cb619b1c1528 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:14.113659 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6dd67e51-9766-4ab3-831e-cb619b1c1528 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:14.119270 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-53ec18fe-f9ca-4b2b-bfe2-071e2fe9b2f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c756cb712bb4310bf2c93abdd1193a8'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:14.120141 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-53ec18fe-f9ca-4b2b-bfe2-071e2fe9b2f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c756cb712bb4310bf2c93abdd1193a8'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:14.126942 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1029/2063] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:29:13 2022] DELETE /identity/v3/users/9c756cb712bb4310bf2c93abdd1193a8 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:14.127435 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:14.181438 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd67e51-9766-4ab3-831e-cb619b1c1528 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:14.182891 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd67e51-9766-4ab3-831e-cb619b1c1528 service cinder] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:14.193128 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1035/2064] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:29:13 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 20:29:14.441733 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec18fe-f9ca-4b2b-bfe2-071e2fe9b2f9 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:14.443432 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-53ec18fe-f9ca-4b2b-bfe2-071e2fe9b2f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:14.476611 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:14.531474 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec18fe-f9ca-4b2b-bfe2-071e2fe9b2f9 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:14.537294 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee1d53f-9d96-417b-b21d-674c901d145a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:14.538003 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1d53f-9d96-417b-b21d-674c901d145a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:14.538289 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1d53f-9d96-417b-b21d-674c901d145a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:14.538566 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1d53f-9d96-417b-b21d-674c901d145a admin admin] PATH_INFO: `/v3/projects/d780c7d3692444f6ba1b9813429a1d21` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:14.539958 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee1d53f-9d96-417b-b21d-674c901d145a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d780c7d3692444f6ba1b9813429a1d21)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:14.542285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee1d53f-9d96-417b-b21d-674c901d145a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:14.600245 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2ee1d53f-9d96-417b-b21d-674c901d145a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd780c7d3692444f6ba1b9813429a1d21', 'request_id': 'req-2ee1d53f-9d96-417b-b21d-674c901d145a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:14.605723 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2ee1d53f-9d96-417b-b21d-674c901d145a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd780c7d3692444f6ba1b9813429a1d21', 'request_id': 'req-2ee1d53f-9d96-417b-b21d-674c901d145a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:14.612207 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2ee1d53f-9d96-417b-b21d-674c901d145a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd780c7d3692444f6ba1b9813429a1d21', 'request_id': 'req-2ee1d53f-9d96-417b-b21d-674c901d145a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:14.625633 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2ee1d53f-9d96-417b-b21d-674c901d145a admin admin] The token cache is being invalidate because project d780c7d3692444f6ba1b9813429a1d21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:29:14.625922 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2ee1d53f-9d96-417b-b21d-674c901d145a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:14.626537 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2ee1d53f-9d96-417b-b21d-674c901d145a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:14.628720 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1030/2065] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:29:14 2022] DELETE /identity/v3/projects/d780c7d3692444f6ba1b9813429a1d21 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:18.697810 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd67e51-9766-4ab3-831e-cb619b1c1528 service cinder] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:18.700083 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6dd67e51-9766-4ab3-831e-cb619b1c1528 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:18.714676 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd67e51-9766-4ab3-831e-cb619b1c1528 service cinder] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:18.775840 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f39e6-18aa-47ba-9bc6-79e0b04f39fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:18.776474 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f39e6-18aa-47ba-9bc6-79e0b04f39fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:18.776749 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f39e6-18aa-47ba-9bc6-79e0b04f39fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:18.777019 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f39e6-18aa-47ba-9bc6-79e0b04f39fc admin admin] PATH_INFO: `/v3/users/001eb149f5134a00be66f5203bc1a287` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:18.778180 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f39e6-18aa-47ba-9bc6-79e0b04f39fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=001eb149f5134a00be66f5203bc1a287)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:18.780160 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f39e6-18aa-47ba-9bc6-79e0b04f39fc admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:18.823728 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-a30f39e6-18aa-47ba-9bc6-79e0b04f39fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '001eb149f5134a00be66f5203bc1a287'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:18.835869 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-a30f39e6-18aa-47ba-9bc6-79e0b04f39fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '001eb149f5134a00be66f5203bc1a287'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:18.842583 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-a30f39e6-18aa-47ba-9bc6-79e0b04f39fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '001eb149f5134a00be66f5203bc1a287'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:18.849506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-a30f39e6-18aa-47ba-9bc6-79e0b04f39fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '001eb149f5134a00be66f5203bc1a287'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:18.850124 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-a30f39e6-18aa-47ba-9bc6-79e0b04f39fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '001eb149f5134a00be66f5203bc1a287'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:18.855855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1036/2066] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:29:18 2022] DELETE /identity/v3/users/001eb149f5134a00be66f5203bc1a287 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:18.860263 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cbaa8-4603-4891-8cc4-db85b8aa06ef None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:18.860737 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cbaa8-4603-4891-8cc4-db85b8aa06ef None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:18.860859 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cbaa8-4603-4891-8cc4-db85b8aa06ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:18.880256 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-8f6cbaa8-4603-4891-8cc4-db85b8aa06ef None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:18.881406 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8f6cbaa8-4603-4891-8cc4-db85b8aa06ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:18.939607 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1031/2067] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:18.949877 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f39e6-18aa-47ba-9bc6-79e0b04f39fc admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:18.951263 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a30f39e6-18aa-47ba-9bc6-79e0b04f39fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:18.986829 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:19.013992 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f39e6-18aa-47ba-9bc6-79e0b04f39fc admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:19.017716 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af6657-685a-4b47-a361-96be5fcbe8d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:19.018317 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af6657-685a-4b47-a361-96be5fcbe8d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:19.018590 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af6657-685a-4b47-a361-96be5fcbe8d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:19.018914 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af6657-685a-4b47-a361-96be5fcbe8d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:19.029055 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6af6657-685a-4b47-a361-96be5fcbe8d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:19.030499 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6af6657-685a-4b47-a361-96be5fcbe8d7 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:19.039926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1037/2068] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:29:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:29:19.405064 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cbaa8-4603-4891-8cc4-db85b8aa06ef None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:19.406651 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8f6cbaa8-4603-4891-8cc4-db85b8aa06ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:19.424458 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cbaa8-4603-4891-8cc4-db85b8aa06ef None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:19.435801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d21abd-672d-439a-b8e8-0afd353b9ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:19.436464 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d21abd-672d-439a-b8e8-0afd353b9ede admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:19.436737 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d21abd-672d-439a-b8e8-0afd353b9ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:19.437003 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d21abd-672d-439a-b8e8-0afd353b9ede admin admin] PATH_INFO: `/v3/projects/971ae822bfd84eacb22fd82a5626d451` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:19.438279 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d21abd-672d-439a-b8e8-0afd353b9ede admin admin] RBAC: Authorizing `identity:delete_project(project_id=971ae822bfd84eacb22fd82a5626d451)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:19.440098 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d21abd-672d-439a-b8e8-0afd353b9ede admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:19.498400 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:19.500057 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-26d21abd-672d-439a-b8e8-0afd353b9ede admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '971ae822bfd84eacb22fd82a5626d451', 'request_id': 'req-26d21abd-672d-439a-b8e8-0afd353b9ede'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:19.515617 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-26d21abd-672d-439a-b8e8-0afd353b9ede admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '971ae822bfd84eacb22fd82a5626d451', 'request_id': 'req-26d21abd-672d-439a-b8e8-0afd353b9ede'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:19.528291 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-26d21abd-672d-439a-b8e8-0afd353b9ede admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '971ae822bfd84eacb22fd82a5626d451', 'request_id': 'req-26d21abd-672d-439a-b8e8-0afd353b9ede'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:19.546929 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-26d21abd-672d-439a-b8e8-0afd353b9ede admin admin] The token cache is being invalidate because project 971ae822bfd84eacb22fd82a5626d451 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:29:19.547227 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-26d21abd-672d-439a-b8e8-0afd353b9ede admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:19.547880 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-26d21abd-672d-439a-b8e8-0afd353b9ede admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:19.549664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1032/2069] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:29:19 2022] DELETE /identity/v3/projects/971ae822bfd84eacb22fd82a5626d451 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:19.554327 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af6657-685a-4b47-a361-96be5fcbe8d7 service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:19.564996 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af6657-685a-4b47-a361-96be5fcbe8d7 service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:19.629237 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72ba1ea-5476-465c-b363-1fa87ad3afc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:19.630094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ba1ea-5476-465c-b363-1fa87ad3afc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:19.630374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ba1ea-5476-465c-b363-1fa87ad3afc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:19.630760 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ba1ea-5476-465c-b363-1fa87ad3afc2 admin admin] PATH_INFO: `/v3/users/e61df4944333486d8cae165c7ae0ed51` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:19.631888 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72ba1ea-5476-465c-b363-1fa87ad3afc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e61df4944333486d8cae165c7ae0ed51)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:19.634306 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72ba1ea-5476-465c-b363-1fa87ad3afc2 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:19.692035 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-c72ba1ea-5476-465c-b363-1fa87ad3afc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e61df4944333486d8cae165c7ae0ed51'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:19.704635 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-c72ba1ea-5476-465c-b363-1fa87ad3afc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e61df4944333486d8cae165c7ae0ed51'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:19.712217 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-c72ba1ea-5476-465c-b363-1fa87ad3afc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e61df4944333486d8cae165c7ae0ed51'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:19.721180 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-c72ba1ea-5476-465c-b363-1fa87ad3afc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e61df4944333486d8cae165c7ae0ed51'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:19.721816 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-c72ba1ea-5476-465c-b363-1fa87ad3afc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e61df4944333486d8cae165c7ae0ed51'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:19.728236 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1038/2070] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:29:19 2022] DELETE /identity/v3/users/e61df4944333486d8cae165c7ae0ed51 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:20.109790 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d21abd-672d-439a-b8e8-0afd353b9ede admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:20.112216 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-26d21abd-672d-439a-b8e8-0afd353b9ede admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:20.130842 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d21abd-672d-439a-b8e8-0afd353b9ede admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:20.201747 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c49a0cb-a38a-4006-bce9-1b12b7dbe9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:20.202415 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49a0cb-a38a-4006-bce9-1b12b7dbe9a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:20.202705 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49a0cb-a38a-4006-bce9-1b12b7dbe9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:20.202987 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49a0cb-a38a-4006-bce9-1b12b7dbe9a4 admin admin] PATH_INFO: `/v3/projects/4b504d4ede2245729eea1bb81f25519b` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:20.204123 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c49a0cb-a38a-4006-bce9-1b12b7dbe9a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b504d4ede2245729eea1bb81f25519b)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:20.206129 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c49a0cb-a38a-4006-bce9-1b12b7dbe9a4 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:20.262716 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4c49a0cb-a38a-4006-bce9-1b12b7dbe9a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b504d4ede2245729eea1bb81f25519b', 'request_id': 'req-4c49a0cb-a38a-4006-bce9-1b12b7dbe9a4'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:20.269269 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4c49a0cb-a38a-4006-bce9-1b12b7dbe9a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b504d4ede2245729eea1bb81f25519b', 'request_id': 'req-4c49a0cb-a38a-4006-bce9-1b12b7dbe9a4'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:20.276419 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4c49a0cb-a38a-4006-bce9-1b12b7dbe9a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b504d4ede2245729eea1bb81f25519b', 'request_id': 'req-4c49a0cb-a38a-4006-bce9-1b12b7dbe9a4'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:20.299576 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4c49a0cb-a38a-4006-bce9-1b12b7dbe9a4 admin admin] The token cache is being invalidate because project 4b504d4ede2245729eea1bb81f25519b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:29:20.299882 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4c49a0cb-a38a-4006-bce9-1b12b7dbe9a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:20.300624 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4c49a0cb-a38a-4006-bce9-1b12b7dbe9a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:20.305665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1033/2071] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:29:20 2022] DELETE /identity/v3/projects/4b504d4ede2245729eea1bb81f25519b => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:20.309151 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72ba1ea-5476-465c-b363-1fa87ad3afc2 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:20.315198 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-c72ba1ea-5476-465c-b363-1fa87ad3afc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:20.339709 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72ba1ea-5476-465c-b363-1fa87ad3afc2 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:20.404683 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:20.416196 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d753481-4c73-43f9-8bac-fd96dcf39d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:20.416894 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d753481-4c73-43f9-8bac-fd96dcf39d86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:20.417238 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d753481-4c73-43f9-8bac-fd96dcf39d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:20.417769 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d753481-4c73-43f9-8bac-fd96dcf39d86 admin admin] PATH_INFO: `/v3/users/0506913789984cef915b0c5f62c906dc` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:20.421386 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d753481-4c73-43f9-8bac-fd96dcf39d86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0506913789984cef915b0c5f62c906dc)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:20.424086 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d753481-4c73-43f9-8bac-fd96dcf39d86 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:20.479222 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-9d753481-4c73-43f9-8bac-fd96dcf39d86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0506913789984cef915b0c5f62c906dc'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:20.491878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-9d753481-4c73-43f9-8bac-fd96dcf39d86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0506913789984cef915b0c5f62c906dc'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:20.499439 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-9d753481-4c73-43f9-8bac-fd96dcf39d86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0506913789984cef915b0c5f62c906dc'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:20.507000 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-9d753481-4c73-43f9-8bac-fd96dcf39d86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0506913789984cef915b0c5f62c906dc'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:20.507624 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-9d753481-4c73-43f9-8bac-fd96dcf39d86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0506913789984cef915b0c5f62c906dc'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:20.513782 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1039/2072] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:29:20 2022] DELETE /identity/v3/users/0506913789984cef915b0c5f62c906dc => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:20.834670 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c49a0cb-a38a-4006-bce9-1b12b7dbe9a4 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:20.836307 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4c49a0cb-a38a-4006-bce9-1b12b7dbe9a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:20.850528 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c49a0cb-a38a-4006-bce9-1b12b7dbe9a4 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:20.908760 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605111b-3d33-4b3e-8b79-c75f7e0e0557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:20.909432 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605111b-3d33-4b3e-8b79-c75f7e0e0557 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:20.909716 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605111b-3d33-4b3e-8b79-c75f7e0e0557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:20.909990 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605111b-3d33-4b3e-8b79-c75f7e0e0557 admin admin] PATH_INFO: `/v3/projects/f0696d1735334419b9c56698bd8a9ff5` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:20.911098 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5605111b-3d33-4b3e-8b79-c75f7e0e0557 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0696d1735334419b9c56698bd8a9ff5)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:20.912996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5605111b-3d33-4b3e-8b79-c75f7e0e0557 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:20.921972 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:20.963737 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5605111b-3d33-4b3e-8b79-c75f7e0e0557 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0696d1735334419b9c56698bd8a9ff5', 'request_id': 'req-5605111b-3d33-4b3e-8b79-c75f7e0e0557'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:20.970021 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5605111b-3d33-4b3e-8b79-c75f7e0e0557 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0696d1735334419b9c56698bd8a9ff5', 'request_id': 'req-5605111b-3d33-4b3e-8b79-c75f7e0e0557'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:20.979267 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5605111b-3d33-4b3e-8b79-c75f7e0e0557 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0696d1735334419b9c56698bd8a9ff5', 'request_id': 'req-5605111b-3d33-4b3e-8b79-c75f7e0e0557'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:20.996601 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5605111b-3d33-4b3e-8b79-c75f7e0e0557 admin admin] The token cache is being invalidate because project f0696d1735334419b9c56698bd8a9ff5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:29:20.996890 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5605111b-3d33-4b3e-8b79-c75f7e0e0557 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:20.997532 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5605111b-3d33-4b3e-8b79-c75f7e0e0557 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:20.999032 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1034/2073] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:29:20 2022] DELETE /identity/v3/projects/f0696d1735334419b9c56698bd8a9ff5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:21.004788 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b0ec0-7a56-4b57-8dc9-9404859d799e None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:21.005278 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b0ec0-7a56-4b57-8dc9-9404859d799e None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:21.005430 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b0ec0-7a56-4b57-8dc9-9404859d799e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:21.024860 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-742b0ec0-7a56-4b57-8dc9-9404859d799e None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:21.026088 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-742b0ec0-7a56-4b57-8dc9-9404859d799e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:21.086152 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1040/2074] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:21.096035 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a798c4-6ed5-4d1a-acba-ce8dea8e779e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:21.096320 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a798c4-6ed5-4d1a-acba-ce8dea8e779e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:21.096597 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a798c4-6ed5-4d1a-acba-ce8dea8e779e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:21.120183 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-c0a798c4-6ed5-4d1a-acba-ce8dea8e779e None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:21.121454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c0a798c4-6ed5-4d1a-acba-ce8dea8e779e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:21.130416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1035/2075] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:21.140266 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b0ec0-7a56-4b57-8dc9-9404859d799e None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:21.154346 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b0ec0-7a56-4b57-8dc9-9404859d799e None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:21.159282 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5fde8b-30ff-47e3-a8ad-929ed32b9a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:21.159893 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fde8b-30ff-47e3-a8ad-929ed32b9a46 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:21.160173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fde8b-30ff-47e3-a8ad-929ed32b9a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:21.160444 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fde8b-30ff-47e3-a8ad-929ed32b9a46 admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:21.161095 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5fde8b-30ff-47e3-a8ad-929ed32b9a46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:29:21.161465 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5fde8b-30ff-47e3-a8ad-929ed32b9a46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:21.163058 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5fde8b-30ff-47e3-a8ad-929ed32b9a46 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:21.175996 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1041/2076] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:29:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:29:21.180397 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a798c4-6ed5-4d1a-acba-ce8dea8e779e None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:21.190173 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a798c4-6ed5-4d1a-acba-ce8dea8e779e None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:21.194928 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee042d-e150-4f47-91eb-fed2468d6431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:21.195601 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee042d-e150-4f47-91eb-fed2468d6431 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:21.195946 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee042d-e150-4f47-91eb-fed2468d6431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:21.196286 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee042d-e150-4f47-91eb-fed2468d6431 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:21.196988 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee042d-e150-4f47-91eb-fed2468d6431 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:21.198979 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee042d-e150-4f47-91eb-fed2468d6431 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:21.231788 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1036/2077] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:29:21 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:29:21.235906 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5fde8b-30ff-47e3-a8ad-929ed32b9a46 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:21.246348 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5fde8b-30ff-47e3-a8ad-929ed32b9a46 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:21.302344 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133928cf-167e-4fa5-9346-3769e0d8a226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:21.302946 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133928cf-167e-4fa5-9346-3769e0d8a226 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:21.303335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133928cf-167e-4fa5-9346-3769e0d8a226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:21.303615 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133928cf-167e-4fa5-9346-3769e0d8a226 admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:21.304250 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133928cf-167e-4fa5-9346-3769e0d8a226 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:21.306209 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133928cf-167e-4fa5-9346-3769e0d8a226 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:21.314077 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:21.329085 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1042/2078] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:29:21 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:29:21.333344 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee042d-e150-4f47-91eb-fed2468d6431 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:21.345340 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee042d-e150-4f47-91eb-fed2468d6431 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:21.349948 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60037ba7-f763-4db7-b1e5-462137d6fa2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:21.350545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60037ba7-f763-4db7-b1e5-462137d6fa2d admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:21.350826 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60037ba7-f763-4db7-b1e5-462137d6fa2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:21.351100 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60037ba7-f763-4db7-b1e5-462137d6fa2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:21.351737 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60037ba7-f763-4db7-b1e5-462137d6fa2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:21.353355 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60037ba7-f763-4db7-b1e5-462137d6fa2d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:21.363046 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1037/2079] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:29:21.367039 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133928cf-167e-4fa5-9346-3769e0d8a226 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:21.376902 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133928cf-167e-4fa5-9346-3769e0d8a226 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:21.382272 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67aaab7-5543-4190-9c23-f9c81a1d97d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:21.382995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67aaab7-5543-4190-9c23-f9c81a1d97d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:21.383273 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67aaab7-5543-4190-9c23-f9c81a1d97d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:21.383552 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67aaab7-5543-4190-9c23-f9c81a1d97d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:21.384244 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67aaab7-5543-4190-9c23-f9c81a1d97d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:21.385933 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67aaab7-5543-4190-9c23-f9c81a1d97d3 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:21.397074 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1043/2080] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:29:21.401298 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60037ba7-f763-4db7-b1e5-462137d6fa2d admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:21.411816 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60037ba7-f763-4db7-b1e5-462137d6fa2d admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:21.416708 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826874bd-b7d7-4e66-a8e8-d14f926e0c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:21.417640 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826874bd-b7d7-4e66-a8e8-d14f926e0c61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:21.417640 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826874bd-b7d7-4e66-a8e8-d14f926e0c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:21.417959 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826874bd-b7d7-4e66-a8e8-d14f926e0c61 admin admin] PATH_INFO: `/v3/projects/476245dd54764377814d117257a8e332/users/5b95b1c83d47492d98474ca1998c689d/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:21.433451 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826874bd-b7d7-4e66-a8e8-d14f926e0c61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=476245dd54764377814d117257a8e332, user_id=5b95b1c83d47492d98474ca1998c689d, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:21.436105 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826874bd-b7d7-4e66-a8e8-d14f926e0c61 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:21.448991 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1038/2081] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:29:21 2022] PUT /identity/v3/projects/476245dd54764377814d117257a8e332/users/5b95b1c83d47492d98474ca1998c689d/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:29:21.460729 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec34ce9-1e7f-4a03-84ba-995accc838c7 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:21.461713 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec34ce9-1e7f-4a03-84ba-995accc838c7 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:21.461997 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec34ce9-1e7f-4a03-84ba-995accc838c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:21.491387 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-2ec34ce9-1e7f-4a03-84ba-995accc838c7 None None] MFA Rules not processed for user `5b95b1c83d47492d98474ca1998c689d`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:21.492484 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-2ec34ce9-1e7f-4a03-84ba-995accc838c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:21.569914 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1044/2082] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:21.575258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826874bd-b7d7-4e66-a8e8-d14f926e0c61 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:21.641739 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826874bd-b7d7-4e66-a8e8-d14f926e0c61 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:21.648514 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5397a7f-5399-4df5-a85d-cbfeba48ff1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:21.649133 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5397a7f-5399-4df5-a85d-cbfeba48ff1b admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:21.649446 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5397a7f-5399-4df5-a85d-cbfeba48ff1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:21.649721 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5397a7f-5399-4df5-a85d-cbfeba48ff1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:21.650370 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5397a7f-5399-4df5-a85d-cbfeba48ff1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:21.652361 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5397a7f-5399-4df5-a85d-cbfeba48ff1b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:21.685284 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1039/2083] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:29:21 2022] POST /identity/v3/projects => generated 367 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:29:21.689267 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec34ce9-1e7f-4a03-84ba-995accc838c7 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:21.701372 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec34ce9-1e7f-4a03-84ba-995accc838c7 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:21.721032 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5397a7f-5399-4df5-a85d-cbfeba48ff1b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:21.723050 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b5397a7f-5399-4df5-a85d-cbfeba48ff1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:21.745524 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:21.783860 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510e0f7-84db-4cf8-9a49-d02bb77a2390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:21.784788 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510e0f7-84db-4cf8-9a49-d02bb77a2390 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:21.785416 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510e0f7-84db-4cf8-9a49-d02bb77a2390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:21.785718 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510e0f7-84db-4cf8-9a49-d02bb77a2390 admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:21.786480 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7510e0f7-84db-4cf8-9a49-d02bb77a2390 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:21.788400 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7510e0f7-84db-4cf8-9a49-d02bb77a2390 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:21.812288 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5397a7f-5399-4df5-a85d-cbfeba48ff1b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:21.814183 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1045/2084] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:29:21 2022] POST /identity/v3/users => generated 290 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:29:21.817556 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80bef8c-a064-4a7c-b0a4-d46f2f03136c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:21.818270 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80bef8c-a064-4a7c-b0a4-d46f2f03136c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:21.818381 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510e0f7-84db-4cf8-9a49-d02bb77a2390 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:21.818822 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80bef8c-a064-4a7c-b0a4-d46f2f03136c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:21.819164 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80bef8c-a064-4a7c-b0a4-d46f2f03136c admin admin] PATH_INFO: `/v3/users/25e8798c64624ca6b0c3a652b6a7681c` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:21.820381 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80bef8c-a064-4a7c-b0a4-d46f2f03136c admin admin] RBAC: Authorizing `identity:delete_user(user_id=25e8798c64624ca6b0c3a652b6a7681c)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:21.822521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80bef8c-a064-4a7c-b0a4-d46f2f03136c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:21.831264 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510e0f7-84db-4cf8-9a49-d02bb77a2390 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:21.836937 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e94cac-729c-4800-9d51-b559bd4b0ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:21.838569 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e94cac-729c-4800-9d51-b559bd4b0ebc admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:21.838918 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e94cac-729c-4800-9d51-b559bd4b0ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:21.839125 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e94cac-729c-4800-9d51-b559bd4b0ebc admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:21.839794 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e94cac-729c-4800-9d51-b559bd4b0ebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:21.841481 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e94cac-729c-4800-9d51-b559bd4b0ebc admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:21.858360 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1046/2085] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:29:21.863583 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e94cac-729c-4800-9d51-b559bd4b0ebc admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:21.874620 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e94cac-729c-4800-9d51-b559bd4b0ebc admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:21.879720 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549a525-4361-4126-aa26-59bb723e92f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:21.880499 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549a525-4361-4126-aa26-59bb723e92f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:21.880781 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549a525-4361-4126-aa26-59bb723e92f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:21.881071 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549a525-4361-4126-aa26-59bb723e92f9 admin admin] PATH_INFO: `/v3/projects/1b81a0dbb56b4f9393b056686381105c/users/5c650434dd444489bfb81a21069f6ea0/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:21.881857 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b80bef8c-a064-4a7c-b0a4-d46f2f03136c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25e8798c64624ca6b0c3a652b6a7681c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:21.890583 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b80bef8c-a064-4a7c-b0a4-d46f2f03136c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25e8798c64624ca6b0c3a652b6a7681c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:21.896809 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a549a525-4361-4126-aa26-59bb723e92f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b81a0dbb56b4f9393b056686381105c, user_id=5c650434dd444489bfb81a21069f6ea0, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:21.899470 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a549a525-4361-4126-aa26-59bb723e92f9 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:21.902752 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b80bef8c-a064-4a7c-b0a4-d46f2f03136c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25e8798c64624ca6b0c3a652b6a7681c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:21.911176 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b80bef8c-a064-4a7c-b0a4-d46f2f03136c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25e8798c64624ca6b0c3a652b6a7681c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:21.911975 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b80bef8c-a064-4a7c-b0a4-d46f2f03136c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25e8798c64624ca6b0c3a652b6a7681c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:21.914211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1047/2086] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:29:21 2022] PUT /identity/v3/projects/1b81a0dbb56b4f9393b056686381105c/users/5c650434dd444489bfb81a21069f6ea0/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:29:21.917782 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549a525-4361-4126-aa26-59bb723e92f9 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:21.918448 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1040/2087] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:29:21 2022] DELETE /identity/v3/users/25e8798c64624ca6b0c3a652b6a7681c => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:29:21.922222 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a549a525-4361-4126-aa26-59bb723e92f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:21.922524 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d3b8c-0897-4383-aa2f-d00d989656fc None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:21.922839 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d3b8c-0897-4383-aa2f-d00d989656fc None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:21.923200 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d3b8c-0897-4383-aa2f-d00d989656fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:21.946216 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-689d3b8c-0897-4383-aa2f-d00d989656fc None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:21.947671 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-689d3b8c-0897-4383-aa2f-d00d989656fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:21.948254 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:21.990322 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549a525-4361-4126-aa26-59bb723e92f9 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:22.007605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1041/2088] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:29:22.017636 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d3b8c-0897-4383-aa2f-d00d989656fc None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:22.019011 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-689d3b8c-0897-4383-aa2f-d00d989656fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:22.047841 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6d36e-cdb3-4123-8769-dcfa2c981d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:22.048542 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6d36e-cdb3-4123-8769-dcfa2c981d8a admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:22.048721 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6d36e-cdb3-4123-8769-dcfa2c981d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:22.048992 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6d36e-cdb3-4123-8769-dcfa2c981d8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:22.049651 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f6d36e-cdb3-4123-8769-dcfa2c981d8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:22.051245 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f6d36e-cdb3-4123-8769-dcfa2c981d8a admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:22.060740 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:22.061621 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1048/2089] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:29:22.065703 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6d36e-cdb3-4123-8769-dcfa2c981d8a admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:22.075545 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6d36e-cdb3-4123-8769-dcfa2c981d8a admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:22.077499 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d3b8c-0897-4383-aa2f-d00d989656fc None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:22.080922 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea42255-b0f0-44d6-be1b-91bb9addbbc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:22.081486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf06db-eea7-4805-bdd0-57e1d38bfc2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:22.081720 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea42255-b0f0-44d6-be1b-91bb9addbbc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:22.082004 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea42255-b0f0-44d6-be1b-91bb9addbbc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:22.082186 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf06db-eea7-4805-bdd0-57e1d38bfc2c service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:22.082474 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf06db-eea7-4805-bdd0-57e1d38bfc2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:22.082729 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea42255-b0f0-44d6-be1b-91bb9addbbc9 admin admin] PATH_INFO: `/v3/projects/1b81a0dbb56b4f9393b056686381105c/users/5c650434dd444489bfb81a21069f6ea0/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:22.083012 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf06db-eea7-4805-bdd0-57e1d38bfc2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:22.085056 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea42255-b0f0-44d6-be1b-91bb9addbbc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b81a0dbb56b4f9393b056686381105c, user_id=5c650434dd444489bfb81a21069f6ea0, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:22.087774 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea42255-b0f0-44d6-be1b-91bb9addbbc9 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:22.093083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddf06db-eea7-4805-bdd0-57e1d38bfc2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:22.094738 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddf06db-eea7-4805-bdd0-57e1d38bfc2c service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:22.099057 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1049/2090] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:29:22 2022] PUT /identity/v3/projects/1b81a0dbb56b4f9393b056686381105c/users/5c650434dd444489bfb81a21069f6ea0/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:29:22.107425 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1042/2091] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:29:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 20:29:22.110973 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b583b-f64a-4047-a4e4-51ade2cfca65 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:22.111360 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b583b-f64a-4047-a4e4-51ade2cfca65 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:22.111987 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b583b-f64a-4047-a4e4-51ade2cfca65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:22.141955 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-265b583b-f64a-4047-a4e4-51ade2cfca65 None None] MFA Rules not processed for user `5c650434dd444489bfb81a21069f6ea0`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:22.144018 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-265b583b-f64a-4047-a4e4-51ade2cfca65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:22.224074 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1050/2092] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:22 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:22.236860 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf06db-eea7-4805-bdd0-57e1d38bfc2c service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:22.239082 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8ddf06db-eea7-4805-bdd0-57e1d38bfc2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:22.352131 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf06db-eea7-4805-bdd0-57e1d38bfc2c service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:22.448018 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43439f30-79f6-493c-9d5b-dfac0d0ff86a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:22.449948 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43439f30-79f6-493c-9d5b-dfac0d0ff86a service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:22.450317 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43439f30-79f6-493c-9d5b-dfac0d0ff86a service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:22.450746 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43439f30-79f6-493c-9d5b-dfac0d0ff86a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:22.467879 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43439f30-79f6-493c-9d5b-dfac0d0ff86a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:22.472998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43439f30-79f6-493c-9d5b-dfac0d0ff86a service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:22.506690 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265b583b-f64a-4047-a4e4-51ade2cfca65 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:22.510569 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-265b583b-f64a-4047-a4e4-51ade2cfca65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:22.518440 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1043/2093] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:29:22 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:29:22.572630 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:22.598175 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265b583b-f64a-4047-a4e4-51ade2cfca65 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:22.602921 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd836c-9acc-4db9-8f11-fdfdd097fcd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:22.603557 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd836c-9acc-4db9-8f11-fdfdd097fcd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:22.603833 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd836c-9acc-4db9-8f11-fdfdd097fcd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:22.604118 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd836c-9acc-4db9-8f11-fdfdd097fcd5 admin admin] PATH_INFO: `/v3/projects/b3ba09adfbc44f268fc06f37938533c7` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:22.605198 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fd836c-9acc-4db9-8f11-fdfdd097fcd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3ba09adfbc44f268fc06f37938533c7)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:22.607927 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fd836c-9acc-4db9-8f11-fdfdd097fcd5 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:22.611566 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43439f30-79f6-493c-9d5b-dfac0d0ff86a service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:22.620724 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43439f30-79f6-493c-9d5b-dfac0d0ff86a service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:22.624662 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2567919-32fa-4899-8094-0517c964ff00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:22.625383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2567919-32fa-4899-8094-0517c964ff00 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:22.625728 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2567919-32fa-4899-8094-0517c964ff00 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:22.626072 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2567919-32fa-4899-8094-0517c964ff00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:22.627800 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a2567919-32fa-4899-8094-0517c964ff00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:22.670643 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-83fd836c-9acc-4db9-8f11-fdfdd097fcd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3ba09adfbc44f268fc06f37938533c7', 'request_id': 'req-83fd836c-9acc-4db9-8f11-fdfdd097fcd5'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:22.682932 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-83fd836c-9acc-4db9-8f11-fdfdd097fcd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3ba09adfbc44f268fc06f37938533c7', 'request_id': 'req-83fd836c-9acc-4db9-8f11-fdfdd097fcd5'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:22.691091 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-83fd836c-9acc-4db9-8f11-fdfdd097fcd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3ba09adfbc44f268fc06f37938533c7', 'request_id': 'req-83fd836c-9acc-4db9-8f11-fdfdd097fcd5'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:22.703946 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2567919-32fa-4899-8094-0517c964ff00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:22.705544 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2567919-32fa-4899-8094-0517c964ff00 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:22.705810 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-83fd836c-9acc-4db9-8f11-fdfdd097fcd5 admin admin] The token cache is being invalidate because project b3ba09adfbc44f268fc06f37938533c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:29:22.706175 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-83fd836c-9acc-4db9-8f11-fdfdd097fcd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:22.706888 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-83fd836c-9acc-4db9-8f11-fdfdd097fcd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:22.708646 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1051/2094] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:29:22 2022] DELETE /identity/v3/projects/b3ba09adfbc44f268fc06f37938533c7 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:22.712450 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd836c-9acc-4db9-8f11-fdfdd097fcd5 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:22.713896 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-83fd836c-9acc-4db9-8f11-fdfdd097fcd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:22.716911 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1044/2095] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:29:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 20:29:22.784494 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd836c-9acc-4db9-8f11-fdfdd097fcd5 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:22.789381 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc32a1d-36ba-4c28-84d5-51f66f579fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:22.790270 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc32a1d-36ba-4c28-84d5-51f66f579fcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:22.790414 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc32a1d-36ba-4c28-84d5-51f66f579fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:22.790752 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc32a1d-36ba-4c28-84d5-51f66f579fcf admin admin] PATH_INFO: `/v3/users/f69bb42471ad45b7b2bd1b476d781882` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:22.792434 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc32a1d-36ba-4c28-84d5-51f66f579fcf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f69bb42471ad45b7b2bd1b476d781882)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:22.794438 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc32a1d-36ba-4c28-84d5-51f66f579fcf admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:22.888250 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6dc32a1d-36ba-4c28-84d5-51f66f579fcf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f69bb42471ad45b7b2bd1b476d781882'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:22.911083 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6dc32a1d-36ba-4c28-84d5-51f66f579fcf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f69bb42471ad45b7b2bd1b476d781882'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:22.919121 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:22.923189 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6dc32a1d-36ba-4c28-84d5-51f66f579fcf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f69bb42471ad45b7b2bd1b476d781882'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:22.950368 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6dc32a1d-36ba-4c28-84d5-51f66f579fcf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f69bb42471ad45b7b2bd1b476d781882'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:22.951009 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6dc32a1d-36ba-4c28-84d5-51f66f579fcf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f69bb42471ad45b7b2bd1b476d781882'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:22.959537 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1052/2096] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:29:22 2022] DELETE /identity/v3/users/f69bb42471ad45b7b2bd1b476d781882 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:29:23.312063 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2567919-32fa-4899-8094-0517c964ff00 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:23.324933 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2567919-32fa-4899-8094-0517c964ff00 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:23.371851 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:23.395018 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea2f6e-223e-4bf9-87f1-fc1e575e9bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:23.395650 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea2f6e-223e-4bf9-87f1-fc1e575e9bfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:23.395931 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea2f6e-223e-4bf9-87f1-fc1e575e9bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:23.396204 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea2f6e-223e-4bf9-87f1-fc1e575e9bfc admin admin] PATH_INFO: `/v3/projects/2f515f9c4e1941cdab63b75c182fd2c7` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:23.397342 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea2f6e-223e-4bf9-87f1-fc1e575e9bfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f515f9c4e1941cdab63b75c182fd2c7)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:23.399276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea2f6e-223e-4bf9-87f1-fc1e575e9bfc admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:23.444299 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-abea2f6e-223e-4bf9-87f1-fc1e575e9bfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f515f9c4e1941cdab63b75c182fd2c7', 'request_id': 'req-abea2f6e-223e-4bf9-87f1-fc1e575e9bfc'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:23.450544 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-abea2f6e-223e-4bf9-87f1-fc1e575e9bfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f515f9c4e1941cdab63b75c182fd2c7', 'request_id': 'req-abea2f6e-223e-4bf9-87f1-fc1e575e9bfc'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:23.456740 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-abea2f6e-223e-4bf9-87f1-fc1e575e9bfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f515f9c4e1941cdab63b75c182fd2c7', 'request_id': 'req-abea2f6e-223e-4bf9-87f1-fc1e575e9bfc'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:23.468922 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-abea2f6e-223e-4bf9-87f1-fc1e575e9bfc admin admin] The token cache is being invalidate because project 2f515f9c4e1941cdab63b75c182fd2c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:29:23.469206 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-abea2f6e-223e-4bf9-87f1-fc1e575e9bfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:23.469839 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-abea2f6e-223e-4bf9-87f1-fc1e575e9bfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:23.471352 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1045/2097] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:29:23 2022] DELETE /identity/v3/projects/2f515f9c4e1941cdab63b75c182fd2c7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:23.476876 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58797b39-9b93-451c-a4c7-be65fec90957 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:23.477396 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58797b39-9b93-451c-a4c7-be65fec90957 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:23.477499 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58797b39-9b93-451c-a4c7-be65fec90957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:23.498162 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-58797b39-9b93-451c-a4c7-be65fec90957 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:23.499351 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-58797b39-9b93-451c-a4c7-be65fec90957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:23.561196 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1053/2098] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:23.569443 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d094e-e837-48aa-976c-d5a52e389067 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:23.569730 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d094e-e837-48aa-976c-d5a52e389067 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:23.570055 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d094e-e837-48aa-976c-d5a52e389067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:23.589626 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-749d094e-e837-48aa-976c-d5a52e389067 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:23.590825 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-749d094e-e837-48aa-976c-d5a52e389067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:23.598932 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1046/2099] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:23.602990 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58797b39-9b93-451c-a4c7-be65fec90957 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:23.613501 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58797b39-9b93-451c-a4c7-be65fec90957 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:23.618695 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6729309-cc75-4103-b65f-7b908d6cbb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:23.619464 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6729309-cc75-4103-b65f-7b908d6cbb39 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:23.619805 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6729309-cc75-4103-b65f-7b908d6cbb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:23.620155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6729309-cc75-4103-b65f-7b908d6cbb39 admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:23.620806 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6729309-cc75-4103-b65f-7b908d6cbb39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:29:23.621693 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6729309-cc75-4103-b65f-7b908d6cbb39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:23.623283 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6729309-cc75-4103-b65f-7b908d6cbb39 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:23.637279 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1054/2100] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:29:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:29:23.639862 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d094e-e837-48aa-976c-d5a52e389067 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:23.650246 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d094e-e837-48aa-976c-d5a52e389067 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:23.654988 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27303f8-800f-426a-8b68-5f36784eb28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:23.655988 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27303f8-800f-426a-8b68-5f36784eb28e admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:23.655988 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27303f8-800f-426a-8b68-5f36784eb28e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:23.656221 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27303f8-800f-426a-8b68-5f36784eb28e admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:23.656761 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27303f8-800f-426a-8b68-5f36784eb28e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:23.658667 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27303f8-800f-426a-8b68-5f36784eb28e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:23.692856 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1047/2101] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:29:23 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:29:23.699208 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6729309-cc75-4103-b65f-7b908d6cbb39 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:23.709633 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6729309-cc75-4103-b65f-7b908d6cbb39 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:23.766776 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb39f4b-a282-47dc-bf62-b08e88a917a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:23.767380 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb39f4b-a282-47dc-bf62-b08e88a917a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:23.767659 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb39f4b-a282-47dc-bf62-b08e88a917a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:23.767934 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb39f4b-a282-47dc-bf62-b08e88a917a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:23.768549 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb39f4b-a282-47dc-bf62-b08e88a917a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:23.770488 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb39f4b-a282-47dc-bf62-b08e88a917a6 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:23.791110 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1055/2102] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:29:23 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:29:23.795827 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27303f8-800f-426a-8b68-5f36784eb28e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:23.806430 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27303f8-800f-426a-8b68-5f36784eb28e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:23.811664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001de53-af63-4257-8e91-af49534c8945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:23.812257 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001de53-af63-4257-8e91-af49534c8945 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:23.812535 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001de53-af63-4257-8e91-af49534c8945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:23.812805 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001de53-af63-4257-8e91-af49534c8945 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:23.813466 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5001de53-af63-4257-8e91-af49534c8945 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:23.815036 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5001de53-af63-4257-8e91-af49534c8945 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:23.824681 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1048/2103] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:29:23.828586 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb39f4b-a282-47dc-bf62-b08e88a917a6 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:23.838638 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb39f4b-a282-47dc-bf62-b08e88a917a6 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:23.843747 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1220a39f-8e7d-40e5-a0a3-f73c1c73ec53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:23.844328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220a39f-8e7d-40e5-a0a3-f73c1c73ec53 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:23.844602 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220a39f-8e7d-40e5-a0a3-f73c1c73ec53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:23.844868 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220a39f-8e7d-40e5-a0a3-f73c1c73ec53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:23.845525 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1220a39f-8e7d-40e5-a0a3-f73c1c73ec53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:23.847097 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1220a39f-8e7d-40e5-a0a3-f73c1c73ec53 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:23.858079 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1056/2104] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:29:23.862136 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001de53-af63-4257-8e91-af49534c8945 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:23.872172 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001de53-af63-4257-8e91-af49534c8945 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:23.877169 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ab1070-1f79-4047-a60d-253dbc374e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:23.877847 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab1070-1f79-4047-a60d-253dbc374e5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:23.878129 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab1070-1f79-4047-a60d-253dbc374e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:23.878402 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab1070-1f79-4047-a60d-253dbc374e5a admin admin] PATH_INFO: `/v3/projects/24a643ab06e449c7a2057690fb26820e/users/d1c14f8e61d54a37bceff98f2f33cd44/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:23.892605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ab1070-1f79-4047-a60d-253dbc374e5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=24a643ab06e449c7a2057690fb26820e, user_id=d1c14f8e61d54a37bceff98f2f33cd44, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:23.895324 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ab1070-1f79-4047-a60d-253dbc374e5a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:23.908352 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1049/2105] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:29:23 2022] PUT /identity/v3/projects/24a643ab06e449c7a2057690fb26820e/users/d1c14f8e61d54a37bceff98f2f33cd44/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:23.918492 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096661a-d183-4ab9-a9d8-5ada3a0d6199 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:23.918773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096661a-d183-4ab9-a9d8-5ada3a0d6199 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:23.919041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096661a-d183-4ab9-a9d8-5ada3a0d6199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:23.948395 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-0096661a-d183-4ab9-a9d8-5ada3a0d6199 None None] MFA Rules not processed for user `d1c14f8e61d54a37bceff98f2f33cd44`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:23.949435 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-0096661a-d183-4ab9-a9d8-5ada3a0d6199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:23.992609 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:24.017053 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1057/2106] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:24.021594 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ab1070-1f79-4047-a60d-253dbc374e5a admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:24.086624 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ab1070-1f79-4047-a60d-253dbc374e5a admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:24.091998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f880192c-927c-4ed7-9dd1-b7e84dc01904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:24.092616 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f880192c-927c-4ed7-9dd1-b7e84dc01904 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:24.092902 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f880192c-927c-4ed7-9dd1-b7e84dc01904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:24.093177 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f880192c-927c-4ed7-9dd1-b7e84dc01904 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:24.093863 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f880192c-927c-4ed7-9dd1-b7e84dc01904 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:24.095757 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f880192c-927c-4ed7-9dd1-b7e84dc01904 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:24.106222 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:24.126719 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1050/2107] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:29:24 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:29:24.130933 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096661a-d183-4ab9-a9d8-5ada3a0d6199 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:24.141662 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096661a-d183-4ab9-a9d8-5ada3a0d6199 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:24.206907 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bdf9b6-f901-462e-93cf-bd4f9c9ab591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:24.207522 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bdf9b6-f901-462e-93cf-bd4f9c9ab591 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:24.207798 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bdf9b6-f901-462e-93cf-bd4f9c9ab591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:24.208067 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bdf9b6-f901-462e-93cf-bd4f9c9ab591 admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:24.208700 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bdf9b6-f901-462e-93cf-bd4f9c9ab591 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:24.210875 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bdf9b6-f901-462e-93cf-bd4f9c9ab591 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:24.236516 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1058/2108] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:29:24 2022] POST /identity/v3/users => generated 297 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:29:24.240862 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f880192c-927c-4ed7-9dd1-b7e84dc01904 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:24.253140 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f880192c-927c-4ed7-9dd1-b7e84dc01904 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:24.258219 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ca9f5-4d13-40ac-a4c5-02ab4ec8c174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:24.259341 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ca9f5-4d13-40ac-a4c5-02ab4ec8c174 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:24.259905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ca9f5-4d13-40ac-a4c5-02ab4ec8c174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:24.260487 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ca9f5-4d13-40ac-a4c5-02ab4ec8c174 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:24.261413 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ca9f5-4d13-40ac-a4c5-02ab4ec8c174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:24.264814 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ca9f5-4d13-40ac-a4c5-02ab4ec8c174 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:24.280255 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1051/2109] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:29:24.284599 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bdf9b6-f901-462e-93cf-bd4f9c9ab591 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:24.294878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bdf9b6-f901-462e-93cf-bd4f9c9ab591 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:24.302444 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabad7b0-1d1e-4218-8606-9211a2db0004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:24.303655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabad7b0-1d1e-4218-8606-9211a2db0004 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:24.304088 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabad7b0-1d1e-4218-8606-9211a2db0004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:24.304654 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabad7b0-1d1e-4218-8606-9211a2db0004 admin admin] PATH_INFO: `/v3/projects/9e9f871548c04966b28a402269fea7cc/users/3bfde02f902c45e88ab306b39f33c946/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:24.334280 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabad7b0-1d1e-4218-8606-9211a2db0004 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e9f871548c04966b28a402269fea7cc, user_id=3bfde02f902c45e88ab306b39f33c946, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:24.337662 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabad7b0-1d1e-4218-8606-9211a2db0004 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:24.359916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1059/2110] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:29:24 2022] PUT /identity/v3/projects/9e9f871548c04966b28a402269fea7cc/users/3bfde02f902c45e88ab306b39f33c946/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:24.365891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ca9f5-4d13-40ac-a4c5-02ab4ec8c174 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:24.387013 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ca9f5-4d13-40ac-a4c5-02ab4ec8c174 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:24.514229 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4bf9ce-9d23-410f-a0d8-a1522609756c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:24.515675 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4bf9ce-9d23-410f-a0d8-a1522609756c admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:24.516163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4bf9ce-9d23-410f-a0d8-a1522609756c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:24.517087 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4bf9ce-9d23-410f-a0d8-a1522609756c admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:24.518428 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4bf9ce-9d23-410f-a0d8-a1522609756c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:24.522474 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4bf9ce-9d23-410f-a0d8-a1522609756c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:24.537813 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1052/2111] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:24 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:29:24.542272 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabad7b0-1d1e-4218-8606-9211a2db0004 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:24.555454 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabad7b0-1d1e-4218-8606-9211a2db0004 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:24.562573 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b39d7-2ed7-4343-ab85-3d4ae3bebc29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:24.563324 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b39d7-2ed7-4343-ab85-3d4ae3bebc29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:24.563741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b39d7-2ed7-4343-ab85-3d4ae3bebc29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:24.563982 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b39d7-2ed7-4343-ab85-3d4ae3bebc29 admin admin] PATH_INFO: `/v3/projects/9e9f871548c04966b28a402269fea7cc/users/3bfde02f902c45e88ab306b39f33c946/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:24.566598 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312b39d7-2ed7-4343-ab85-3d4ae3bebc29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e9f871548c04966b28a402269fea7cc, user_id=3bfde02f902c45e88ab306b39f33c946, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:24.569879 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312b39d7-2ed7-4343-ab85-3d4ae3bebc29 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:24.585888 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1060/2112] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:29:24 2022] PUT /identity/v3/projects/9e9f871548c04966b28a402269fea7cc/users/3bfde02f902c45e88ab306b39f33c946/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:24.597881 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff3f4b-82eb-47bd-9dc3-7518384f850f None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:24.598240 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff3f4b-82eb-47bd-9dc3-7518384f850f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:24.598436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff3f4b-82eb-47bd-9dc3-7518384f850f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:24.634165 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-5dff3f4b-82eb-47bd-9dc3-7518384f850f None None] MFA Rules not processed for user `3bfde02f902c45e88ab306b39f33c946`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:24.635461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5dff3f4b-82eb-47bd-9dc3-7518384f850f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:24.720733 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1053/2113] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:24 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:24.732516 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b39d7-2ed7-4343-ab85-3d4ae3bebc29 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:24.734332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-312b39d7-2ed7-4343-ab85-3d4ae3bebc29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:24.805534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b39d7-2ed7-4343-ab85-3d4ae3bebc29 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:24.810293 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c06ecb-3004-43dd-adf4-1c497e85b1df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:24.810711 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c06ecb-3004-43dd-adf4-1c497e85b1df service cinder] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:24.811060 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c06ecb-3004-43dd-adf4-1c497e85b1df service cinder] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:24.811404 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c06ecb-3004-43dd-adf4-1c497e85b1df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:24.821401 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c06ecb-3004-43dd-adf4-1c497e85b1df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:24.823097 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c06ecb-3004-43dd-adf4-1c497e85b1df service cinder] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:24.834862 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1061/2114] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:29:24 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:29:24.949934 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff3f4b-82eb-47bd-9dc3-7518384f850f None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:24.959252 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff3f4b-82eb-47bd-9dc3-7518384f850f None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:24.962984 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64599ee8-3338-441c-bc38-8d7e5bc605e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:24.963590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64599ee8-3338-441c-bc38-8d7e5bc605e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:24.963878 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64599ee8-3338-441c-bc38-8d7e5bc605e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:24.964133 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64599ee8-3338-441c-bc38-8d7e5bc605e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:25.021014 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64599ee8-3338-441c-bc38-8d7e5bc605e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:25.022475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64599ee8-3338-441c-bc38-8d7e5bc605e8 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:25.026003 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:25.032884 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1054/2115] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:29:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:29:27.813708 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c06ecb-3004-43dd-adf4-1c497e85b1df service cinder] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:27.887487 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:27.959231 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c06ecb-3004-43dd-adf4-1c497e85b1df service cinder] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:27.971587 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b886ca-4112-4733-b6d1-af1e0929f9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:27.973734 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b886ca-4112-4733-b6d1-af1e0929f9e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:27.974258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b886ca-4112-4733-b6d1-af1e0929f9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:27.975042 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b886ca-4112-4733-b6d1-af1e0929f9e3 admin admin] PATH_INFO: `/v3/users/d1c14f8e61d54a37bceff98f2f33cd44` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:27.977119 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b886ca-4112-4733-b6d1-af1e0929f9e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1c14f8e61d54a37bceff98f2f33cd44)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:27.980924 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b886ca-4112-4733-b6d1-af1e0929f9e3 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:28.019154 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64599ee8-3338-441c-bc38-8d7e5bc605e8 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:28.020542 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-64599ee8-3338-441c-bc38-8d7e5bc605e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:28.039465 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64599ee8-3338-441c-bc38-8d7e5bc605e8 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:28.051479 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e51cb1d-a51d-4937-91b1-e5a0de009ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:28.053015 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51cb1d-a51d-4937-91b1-e5a0de009ff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:28.053465 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51cb1d-a51d-4937-91b1-e5a0de009ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:28.053740 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51cb1d-a51d-4937-91b1-e5a0de009ff3 admin admin] PATH_INFO: `/v3/users/5b95b1c83d47492d98474ca1998c689d` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:28.056162 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e51cb1d-a51d-4937-91b1-e5a0de009ff3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b95b1c83d47492d98474ca1998c689d)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:28.060729 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-61b886ca-4112-4733-b6d1-af1e0929f9e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1c14f8e61d54a37bceff98f2f33cd44'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:28.063127 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e51cb1d-a51d-4937-91b1-e5a0de009ff3 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:28.087697 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-61b886ca-4112-4733-b6d1-af1e0929f9e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1c14f8e61d54a37bceff98f2f33cd44'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:28.098571 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-61b886ca-4112-4733-b6d1-af1e0929f9e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1c14f8e61d54a37bceff98f2f33cd44'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:28.110165 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-61b886ca-4112-4733-b6d1-af1e0929f9e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1c14f8e61d54a37bceff98f2f33cd44'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:28.111548 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-61b886ca-4112-4733-b6d1-af1e0929f9e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1c14f8e61d54a37bceff98f2f33cd44'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:28.122351 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1062/2116] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:29:27 2022] DELETE /identity/v3/users/d1c14f8e61d54a37bceff98f2f33cd44 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:28.126795 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d3600-43c9-4256-baa9-2cb4318b3282 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:28.127556 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d3600-43c9-4256-baa9-2cb4318b3282 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:28.127839 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d3600-43c9-4256-baa9-2cb4318b3282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:28.154233 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-030d3600-43c9-4256-baa9-2cb4318b3282 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:28.157634 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-030d3600-43c9-4256-baa9-2cb4318b3282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:28.177161 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0e51cb1d-a51d-4937-91b1-e5a0de009ff3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b95b1c83d47492d98474ca1998c689d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:28.188826 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0e51cb1d-a51d-4937-91b1-e5a0de009ff3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b95b1c83d47492d98474ca1998c689d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:28.216590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0e51cb1d-a51d-4937-91b1-e5a0de009ff3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b95b1c83d47492d98474ca1998c689d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:28.225970 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0e51cb1d-a51d-4937-91b1-e5a0de009ff3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b95b1c83d47492d98474ca1998c689d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:28.226606 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0e51cb1d-a51d-4937-91b1-e5a0de009ff3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b95b1c83d47492d98474ca1998c689d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:28.244441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1055/2117] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:29:28 2022] DELETE /identity/v3/users/5b95b1c83d47492d98474ca1998c689d => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:29:28.244441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82678ce7-1c39-4e49-8eba-e7c1536e3044 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:28.244441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82678ce7-1c39-4e49-8eba-e7c1536e3044 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:28.244441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82678ce7-1c39-4e49-8eba-e7c1536e3044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:28.273474 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-82678ce7-1c39-4e49-8eba-e7c1536e3044 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:28.278754 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1063/2118] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:28.279701 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-82678ce7-1c39-4e49-8eba-e7c1536e3044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:28.289102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d3600-43c9-4256-baa9-2cb4318b3282 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:28.291833 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-030d3600-43c9-4256-baa9-2cb4318b3282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:28.392113 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d3600-43c9-4256-baa9-2cb4318b3282 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:28.400654 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1056/2119] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:29:28.405461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:28.423506 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82678ce7-1c39-4e49-8eba-e7c1536e3044 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:28.461899 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe3845-1438-4f03-b8ff-69dfcd4b78fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:28.462564 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe3845-1438-4f03-b8ff-69dfcd4b78fd service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:28.462836 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe3845-1438-4f03-b8ff-69dfcd4b78fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:28.463114 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe3845-1438-4f03-b8ff-69dfcd4b78fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:28.474508 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe3845-1438-4f03-b8ff-69dfcd4b78fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:28.476104 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe3845-1438-4f03-b8ff-69dfcd4b78fd service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:28.487589 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82678ce7-1c39-4e49-8eba-e7c1536e3044 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:28.489680 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1064/2120] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:29:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:29:28.491756 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc748ff-a82c-4531-955d-b205b067115b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:28.492494 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc748ff-a82c-4531-955d-b205b067115b service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:28.492984 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc748ff-a82c-4531-955d-b205b067115b service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:28.493438 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc748ff-a82c-4531-955d-b205b067115b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:28.504377 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc748ff-a82c-4531-955d-b205b067115b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:28.506550 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc748ff-a82c-4531-955d-b205b067115b service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:28.527928 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1057/2121] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:29:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 20:29:28.623089 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a4da1-2a43-4934-9115-094af99ce81c None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:28.624220 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a4da1-2a43-4934-9115-094af99ce81c None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:28.624506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a4da1-2a43-4934-9115-094af99ce81c None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:28.626212 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1065/2122] 10.222.0.26 () {54 vars in 921 bytes} [Fri Sep 9 20:29:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:29:28.659464 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9de1b5-e896-46b1-987b-205c7ec29174 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:28.659751 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9de1b5-e896-46b1-987b-205c7ec29174 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:28.660025 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9de1b5-e896-46b1-987b-205c7ec29174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:28.698319 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ec9de1b5-e896-46b1-987b-205c7ec29174 None None] MFA Rules not processed for user `26d24cf0a21d491082b0c2f11e749446`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:28.702098 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ec9de1b5-e896-46b1-987b-205c7ec29174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:28.714648 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01a4da1-2a43-4934-9115-094af99ce81c None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:28.718367 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-e01a4da1-2a43-4934-9115-094af99ce81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:28.727856 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:28.751054 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01a4da1-2a43-4934-9115-094af99ce81c None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:28.758604 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c3e5d5-88b4-4fe3-85ea-1004db1a029f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:28.759490 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3e5d5-88b4-4fe3-85ea-1004db1a029f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:28.760934 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3e5d5-88b4-4fe3-85ea-1004db1a029f admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:28.760934 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3e5d5-88b4-4fe3-85ea-1004db1a029f admin admin] PATH_INFO: `/v3/users/fc41cb3dbe3845d794107f068c70db3f` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:28.762974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c3e5d5-88b4-4fe3-85ea-1004db1a029f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc41cb3dbe3845d794107f068c70db3f)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:28.769823 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c3e5d5-88b4-4fe3-85ea-1004db1a029f admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:28.820713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1058/2123] 10.222.0.26 () {58 vars in 1018 bytes} [Fri Sep 9 20:29:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:28.836213 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50a149-0e0c-4370-93df-5ae01b918811 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:28.836605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50a149-0e0c-4370-93df-5ae01b918811 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:28.837275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50a149-0e0c-4370-93df-5ae01b918811 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:28.838705 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1059/2124] 10.222.0.26 () {54 vars in 961 bytes} [Fri Sep 9 20:29:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:29:28.846563 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d6f20-0a03-4ba6-8b16-f5583ecb46c5 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:28.847121 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d6f20-0a03-4ba6-8b16-f5583ecb46c5 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:28.847431 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d6f20-0a03-4ba6-8b16-f5583ecb46c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:28.848237 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-a5c3e5d5-88b4-4fe3-85ea-1004db1a029f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc41cb3dbe3845d794107f068c70db3f'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:28.867282 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-a5c3e5d5-88b4-4fe3-85ea-1004db1a029f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc41cb3dbe3845d794107f068c70db3f'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:28.868941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-522d6f20-0a03-4ba6-8b16-f5583ecb46c5 None None] MFA Rules not processed for user `26d24cf0a21d491082b0c2f11e749446`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:28.870207 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-522d6f20-0a03-4ba6-8b16-f5583ecb46c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:28.876037 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-a5c3e5d5-88b4-4fe3-85ea-1004db1a029f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc41cb3dbe3845d794107f068c70db3f'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:28.879091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1060/2125] 10.222.0.26 () {58 vars in 1058 bytes} [Fri Sep 9 20:29:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:29:28.884773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-a5c3e5d5-88b4-4fe3-85ea-1004db1a029f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc41cb3dbe3845d794107f068c70db3f'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:28.885459 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-a5c3e5d5-88b4-4fe3-85ea-1004db1a029f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc41cb3dbe3845d794107f068c70db3f'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:28.887966 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d6f20-0a03-4ba6-8b16-f5583ecb46c5 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:28.894186 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1066/2126] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:29:28 2022] DELETE /identity/v3/users/fc41cb3dbe3845d794107f068c70db3f => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:29:28.898620 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d6f20-0a03-4ba6-8b16-f5583ecb46c5 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:28.974732 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5f796-1ac3-42c9-aa05-f8710349fc42 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26d24cf0a21d491082b0c2f11e749446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:28.975803 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5f796-1ac3-42c9-aa05-f8710349fc42 service placement] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:28.976355 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5f796-1ac3-42c9-aa05-f8710349fc42 service placement] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:28.976872 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5f796-1ac3-42c9-aa05-f8710349fc42 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:28.979830 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-22b5f796-1ac3-42c9-aa05-f8710349fc42 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:28.997707 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b5f796-1ac3-42c9-aa05-f8710349fc42 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:29.000805 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b5f796-1ac3-42c9-aa05-f8710349fc42 service placement] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:29.021494 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1061/2127] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:29:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 20:29:29.171294 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c3e5d5-88b4-4fe3-85ea-1004db1a029f admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:29.172975 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a5c3e5d5-88b4-4fe3-85ea-1004db1a029f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:29.232433 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5f796-1ac3-42c9-aa05-f8710349fc42 service placement] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:29.234025 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-22b5f796-1ac3-42c9-aa05-f8710349fc42 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:29.278254 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c3e5d5-88b4-4fe3-85ea-1004db1a029f admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:29.283718 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbded98-0ebc-4f01-8a86-f72de2a478fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:29.284953 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbded98-0ebc-4f01-8a86-f72de2a478fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:29.285469 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbded98-0ebc-4f01-8a86-f72de2a478fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:29.285751 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbded98-0ebc-4f01-8a86-f72de2a478fc admin admin] PATH_INFO: `/v3/projects/95899c2c318f459bb09bad337d79c1a2` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:29.287046 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbded98-0ebc-4f01-8a86-f72de2a478fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=95899c2c318f459bb09bad337d79c1a2)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:29.289443 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbded98-0ebc-4f01-8a86-f72de2a478fc admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:29.318947 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5f796-1ac3-42c9-aa05-f8710349fc42 service placement] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:29.323823 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca9a04-c5ad-435d-a778-034d9ccca974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:29.324451 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca9a04-c5ad-435d-a778-034d9ccca974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:29.324740 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca9a04-c5ad-435d-a778-034d9ccca974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:29.325014 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca9a04-c5ad-435d-a778-034d9ccca974 admin admin] PATH_INFO: `/v3/projects/24a643ab06e449c7a2057690fb26820e` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:29.326464 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ca9a04-c5ad-435d-a778-034d9ccca974 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24a643ab06e449c7a2057690fb26820e)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:29.327613 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:29.328929 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ca9a04-c5ad-435d-a778-034d9ccca974 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:29.348166 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6fbded98-0ebc-4f01-8a86-f72de2a478fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95899c2c318f459bb09bad337d79c1a2', 'request_id': 'req-6fbded98-0ebc-4f01-8a86-f72de2a478fc'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:29.355805 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6fbded98-0ebc-4f01-8a86-f72de2a478fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95899c2c318f459bb09bad337d79c1a2', 'request_id': 'req-6fbded98-0ebc-4f01-8a86-f72de2a478fc'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:29.361786 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6fbded98-0ebc-4f01-8a86-f72de2a478fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95899c2c318f459bb09bad337d79c1a2', 'request_id': 'req-6fbded98-0ebc-4f01-8a86-f72de2a478fc'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:29.373310 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6fbded98-0ebc-4f01-8a86-f72de2a478fc admin admin] The token cache is being invalidate because project 95899c2c318f459bb09bad337d79c1a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:29:29.373734 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6fbded98-0ebc-4f01-8a86-f72de2a478fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:29.373847 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:29.374514 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6fbded98-0ebc-4f01-8a86-f72de2a478fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:29.376277 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1067/2128] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:29:29 2022] DELETE /identity/v3/projects/95899c2c318f459bb09bad337d79c1a2 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:29.376838 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbded98-0ebc-4f01-8a86-f72de2a478fc admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:29.378129 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6fbded98-0ebc-4f01-8a86-f72de2a478fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:29.385504 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a7ca9a04-c5ad-435d-a778-034d9ccca974 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24a643ab06e449c7a2057690fb26820e', 'request_id': 'req-a7ca9a04-c5ad-435d-a778-034d9ccca974'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:29.392606 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a7ca9a04-c5ad-435d-a778-034d9ccca974 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24a643ab06e449c7a2057690fb26820e', 'request_id': 'req-a7ca9a04-c5ad-435d-a778-034d9ccca974'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:29.400197 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a7ca9a04-c5ad-435d-a778-034d9ccca974 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24a643ab06e449c7a2057690fb26820e', 'request_id': 'req-a7ca9a04-c5ad-435d-a778-034d9ccca974'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:29.413961 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a7ca9a04-c5ad-435d-a778-034d9ccca974 admin admin] The token cache is being invalidate because project 24a643ab06e449c7a2057690fb26820e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:29:29.414250 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a7ca9a04-c5ad-435d-a778-034d9ccca974 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:29.414864 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a7ca9a04-c5ad-435d-a778-034d9ccca974 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:29.416374 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1062/2129] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:29:29 2022] DELETE /identity/v3/projects/24a643ab06e449c7a2057690fb26820e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:29:29.420953 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f71e6-0355-45f1-8c1c-46cf2c409ff1 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:29.421262 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f71e6-0355-45f1-8c1c-46cf2c409ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:29.421539 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f71e6-0355-45f1-8c1c-46cf2c409ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:29.441389 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ed5f71e6-0355-45f1-8c1c-46cf2c409ff1 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:29.442582 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ed5f71e6-0355-45f1-8c1c-46cf2c409ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:29.443612 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbded98-0ebc-4f01-8a86-f72de2a478fc admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:29.500682 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab33dd5-c599-41a4-af6b-0c81a8b67ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:29.501351 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab33dd5-c599-41a4-af6b-0c81a8b67ded admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:29.501636 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab33dd5-c599-41a4-af6b-0c81a8b67ded admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:29.501911 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab33dd5-c599-41a4-af6b-0c81a8b67ded admin admin] PATH_INFO: `/v3/projects/476245dd54764377814d117257a8e332` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:29.503053 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab33dd5-c599-41a4-af6b-0c81a8b67ded admin admin] RBAC: Authorizing `identity:delete_project(project_id=476245dd54764377814d117257a8e332)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:29.505101 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab33dd5-c599-41a4-af6b-0c81a8b67ded admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:29.506600 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1063/2130] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:29.507441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f71e6-0355-45f1-8c1c-46cf2c409ff1 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:29.508682 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ed5f71e6-0355-45f1-8c1c-46cf2c409ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:29.520698 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f71e6-0355-45f1-8c1c-46cf2c409ff1 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:29.525204 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f151310a-fc4d-42fa-954e-a0081904543f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:29.525852 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151310a-fc4d-42fa-954e-a0081904543f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:29.526133 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151310a-fc4d-42fa-954e-a0081904543f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:29.526410 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151310a-fc4d-42fa-954e-a0081904543f admin admin] PATH_INFO: `/v3/users/3bfde02f902c45e88ab306b39f33c946` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:29.527595 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f151310a-fc4d-42fa-954e-a0081904543f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bfde02f902c45e88ab306b39f33c946)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:29.529574 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f151310a-fc4d-42fa-954e-a0081904543f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:29.554867 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6ab33dd5-c599-41a4-af6b-0c81a8b67ded admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '476245dd54764377814d117257a8e332', 'request_id': 'req-6ab33dd5-c599-41a4-af6b-0c81a8b67ded'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:29.563170 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6ab33dd5-c599-41a4-af6b-0c81a8b67ded admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '476245dd54764377814d117257a8e332', 'request_id': 'req-6ab33dd5-c599-41a4-af6b-0c81a8b67ded'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:29.570811 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6ab33dd5-c599-41a4-af6b-0c81a8b67ded admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '476245dd54764377814d117257a8e332', 'request_id': 'req-6ab33dd5-c599-41a4-af6b-0c81a8b67ded'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:29.575587 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:29.587410 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6ab33dd5-c599-41a4-af6b-0c81a8b67ded admin admin] The token cache is being invalidate because project 476245dd54764377814d117257a8e332 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:29:29.588053 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6ab33dd5-c599-41a4-af6b-0c81a8b67ded admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:29.588929 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6ab33dd5-c599-41a4-af6b-0c81a8b67ded admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:29.590524 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f151310a-fc4d-42fa-954e-a0081904543f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bfde02f902c45e88ab306b39f33c946'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:29.591697 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1068/2131] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:29:29 2022] DELETE /identity/v3/projects/476245dd54764377814d117257a8e332 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:29.593062 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aaa4a8-a105-452a-b50c-35af75fda5b7 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:29.593230 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aaa4a8-a105-452a-b50c-35af75fda5b7 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:29.593564 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aaa4a8-a105-452a-b50c-35af75fda5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:29.599294 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f151310a-fc4d-42fa-954e-a0081904543f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bfde02f902c45e88ab306b39f33c946'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:29.614213 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f151310a-fc4d-42fa-954e-a0081904543f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bfde02f902c45e88ab306b39f33c946'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:29.614439 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-88aaa4a8-a105-452a-b50c-35af75fda5b7 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:29.617351 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-88aaa4a8-a105-452a-b50c-35af75fda5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:29.622697 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f151310a-fc4d-42fa-954e-a0081904543f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bfde02f902c45e88ab306b39f33c946'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:29.623375 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f151310a-fc4d-42fa-954e-a0081904543f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bfde02f902c45e88ab306b39f33c946'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:29.632803 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1064/2132] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:29:29 2022] DELETE /identity/v3/users/3bfde02f902c45e88ab306b39f33c946 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:29.632803 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f151310a-fc4d-42fa-954e-a0081904543f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:29.632803 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f151310a-fc4d-42fa-954e-a0081904543f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:29.696721 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:29.702534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1069/2133] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:29.706725 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aaa4a8-a105-452a-b50c-35af75fda5b7 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:29.719750 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aaa4a8-a105-452a-b50c-35af75fda5b7 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:29.726674 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc80c83-72ef-44b1-9f56-0039fbff8023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:29.727672 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc80c83-72ef-44b1-9f56-0039fbff8023 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:29.728266 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc80c83-72ef-44b1-9f56-0039fbff8023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:29.728742 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc80c83-72ef-44b1-9f56-0039fbff8023 admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:29.729630 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc80c83-72ef-44b1-9f56-0039fbff8023 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:29:29.729938 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc80c83-72ef-44b1-9f56-0039fbff8023 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:29.733211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc80c83-72ef-44b1-9f56-0039fbff8023 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:29.735220 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f151310a-fc4d-42fa-954e-a0081904543f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:29.742298 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a095d-4b1f-47ce-9072-6191f2dd754f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:29.743465 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a095d-4b1f-47ce-9072-6191f2dd754f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:29.744005 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a095d-4b1f-47ce-9072-6191f2dd754f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:29.744494 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a095d-4b1f-47ce-9072-6191f2dd754f admin admin] PATH_INFO: `/v3/users/5c650434dd444489bfb81a21069f6ea0` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:29.746782 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229a095d-4b1f-47ce-9072-6191f2dd754f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c650434dd444489bfb81a21069f6ea0)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:29.750000 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1070/2134] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:29:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:29:29.753452 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229a095d-4b1f-47ce-9072-6191f2dd754f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:29.770061 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc80c83-72ef-44b1-9f56-0039fbff8023 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:29.783925 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc80c83-72ef-44b1-9f56-0039fbff8023 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:29.790041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d655d-4a84-4a16-a3b9-a60a2be5b1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:29.790742 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d655d-4a84-4a16-a3b9-a60a2be5b1b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:29.791094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d655d-4a84-4a16-a3b9-a60a2be5b1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:29.791413 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d655d-4a84-4a16-a3b9-a60a2be5b1b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:29.792181 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d655d-4a84-4a16-a3b9-a60a2be5b1b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:29.794634 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d655d-4a84-4a16-a3b9-a60a2be5b1b3 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:29.814795 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-229a095d-4b1f-47ce-9072-6191f2dd754f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c650434dd444489bfb81a21069f6ea0'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:29.822695 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-229a095d-4b1f-47ce-9072-6191f2dd754f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c650434dd444489bfb81a21069f6ea0'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:29.831936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1071/2135] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:29:29 2022] POST /identity/v3/projects => generated 375 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:29:29.835741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d655d-4a84-4a16-a3b9-a60a2be5b1b3 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:29.841221 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-229a095d-4b1f-47ce-9072-6191f2dd754f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c650434dd444489bfb81a21069f6ea0'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:29.850904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-229a095d-4b1f-47ce-9072-6191f2dd754f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c650434dd444489bfb81a21069f6ea0'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:29.851593 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-229a095d-4b1f-47ce-9072-6191f2dd754f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c650434dd444489bfb81a21069f6ea0'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:29.859751 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1065/2136] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:29:29 2022] DELETE /identity/v3/users/5c650434dd444489bfb81a21069f6ea0 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:29.909061 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d655d-4a84-4a16-a3b9-a60a2be5b1b3 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:29.965929 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a095d-4b1f-47ce-9072-6191f2dd754f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:29.967655 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-229a095d-4b1f-47ce-9072-6191f2dd754f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:29.988593 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028155c-2731-4060-b903-97b0b871ee76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:29.989614 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028155c-2731-4060-b903-97b0b871ee76 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:29.990173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028155c-2731-4060-b903-97b0b871ee76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:29.990512 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028155c-2731-4060-b903-97b0b871ee76 admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:29.991226 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7028155c-2731-4060-b903-97b0b871ee76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:29.993859 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7028155c-2731-4060-b903-97b0b871ee76 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:30.018670 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:30.028433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1072/2137] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:29:29 2022] POST /identity/v3/users => generated 288 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:29:30.032572 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028155c-2731-4060-b903-97b0b871ee76 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:30.037515 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7028155c-2731-4060-b903-97b0b871ee76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:30.051128 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028155c-2731-4060-b903-97b0b871ee76 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:30.054835 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a095d-4b1f-47ce-9072-6191f2dd754f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:30.055820 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebe4fa4-8076-48a5-a164-ccbacb7d7809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:30.056426 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe4fa4-8076-48a5-a164-ccbacb7d7809 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:30.056875 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe4fa4-8076-48a5-a164-ccbacb7d7809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:30.057307 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe4fa4-8076-48a5-a164-ccbacb7d7809 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:30.057976 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebe4fa4-8076-48a5-a164-ccbacb7d7809 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:30.060115 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebe4fa4-8076-48a5-a164-ccbacb7d7809 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:30.062329 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b9c5c5-5395-43fb-bf66-c9a884f9bf84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:30.063181 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9c5c5-5395-43fb-bf66-c9a884f9bf84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:30.063652 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9c5c5-5395-43fb-bf66-c9a884f9bf84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:30.063982 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9c5c5-5395-43fb-bf66-c9a884f9bf84 admin admin] PATH_INFO: `/v3/projects/9e9f871548c04966b28a402269fea7cc` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:30.065127 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b9c5c5-5395-43fb-bf66-c9a884f9bf84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e9f871548c04966b28a402269fea7cc)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:30.067086 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b9c5c5-5395-43fb-bf66-c9a884f9bf84 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:30.070703 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1073/2138] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:29:30.074400 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebe4fa4-8076-48a5-a164-ccbacb7d7809 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:30.085578 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebe4fa4-8076-48a5-a164-ccbacb7d7809 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:30.092382 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389ae592-e6cb-4045-afe2-f88481dc730a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:30.093008 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ae592-e6cb-4045-afe2-f88481dc730a admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:30.093333 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ae592-e6cb-4045-afe2-f88481dc730a admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:30.093609 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ae592-e6cb-4045-afe2-f88481dc730a admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:30.094281 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389ae592-e6cb-4045-afe2-f88481dc730a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:30.095920 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389ae592-e6cb-4045-afe2-f88481dc730a admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:30.112368 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1074/2139] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:29:30.114495 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389ae592-e6cb-4045-afe2-f88481dc730a admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:30.129649 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389ae592-e6cb-4045-afe2-f88481dc730a admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:30.137424 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c5b9c5c5-5395-43fb-bf66-c9a884f9bf84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e9f871548c04966b28a402269fea7cc', 'request_id': 'req-c5b9c5c5-5395-43fb-bf66-c9a884f9bf84'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:30.144433 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:30.148428 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c5b9c5c5-5395-43fb-bf66-c9a884f9bf84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e9f871548c04966b28a402269fea7cc', 'request_id': 'req-c5b9c5c5-5395-43fb-bf66-c9a884f9bf84'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:30.159726 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c5b9c5c5-5395-43fb-bf66-c9a884f9bf84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e9f871548c04966b28a402269fea7cc', 'request_id': 'req-c5b9c5c5-5395-43fb-bf66-c9a884f9bf84'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:30.179285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c5b9c5c5-5395-43fb-bf66-c9a884f9bf84 admin admin] The token cache is being invalidate because project 9e9f871548c04966b28a402269fea7cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:29:30.179588 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c5b9c5c5-5395-43fb-bf66-c9a884f9bf84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:30.180260 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c5b9c5c5-5395-43fb-bf66-c9a884f9bf84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:30.185126 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1066/2140] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:29:29 2022] DELETE /identity/v3/projects/9e9f871548c04966b28a402269fea7cc => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:30.185126 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b9c5c5-5395-43fb-bf66-c9a884f9bf84 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:30.187155 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c5b9c5c5-5395-43fb-bf66-c9a884f9bf84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:30.203241 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c316b4-97c9-431b-a881-7d2ae64c0ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:30.204012 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c316b4-97c9-431b-a881-7d2ae64c0ba2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:30.204205 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c316b4-97c9-431b-a881-7d2ae64c0ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:30.204483 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c316b4-97c9-431b-a881-7d2ae64c0ba2 admin admin] PATH_INFO: `/v3/projects/2e95b9f1ae2142b4b408191b19f1755c/users/8be4a2a8537749849260ca4b4ce7a4df/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:30.221592 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c316b4-97c9-431b-a881-7d2ae64c0ba2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e95b9f1ae2142b4b408191b19f1755c, user_id=8be4a2a8537749849260ca4b4ce7a4df, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:30.224261 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c316b4-97c9-431b-a881-7d2ae64c0ba2 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:30.240500 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1075/2141] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:29:30 2022] PUT /identity/v3/projects/2e95b9f1ae2142b4b408191b19f1755c/users/8be4a2a8537749849260ca4b4ce7a4df/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:30.241767 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dab20b-279d-44eb-8bef-c131b85f1187 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:30.242055 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dab20b-279d-44eb-8bef-c131b85f1187 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:30.242327 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dab20b-279d-44eb-8bef-c131b85f1187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:30.263108 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-d9dab20b-279d-44eb-8bef-c131b85f1187 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:30.263619 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b9c5c5-5395-43fb-bf66-c9a884f9bf84 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:30.264302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-d9dab20b-279d-44eb-8bef-c131b85f1187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:30.322931 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd5d4e-10a1-4bc8-ba3a-12e7beeee800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:30.323576 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd5d4e-10a1-4bc8-ba3a-12e7beeee800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:30.324454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd5d4e-10a1-4bc8-ba3a-12e7beeee800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:30.324728 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd5d4e-10a1-4bc8-ba3a-12e7beeee800 admin admin] PATH_INFO: `/v3/projects/1b81a0dbb56b4f9393b056686381105c` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:30.326578 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfd5d4e-10a1-4bc8-ba3a-12e7beeee800 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b81a0dbb56b4f9393b056686381105c)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:30.328848 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfd5d4e-10a1-4bc8-ba3a-12e7beeee800 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:30.337454 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1076/2142] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:30.341188 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f838f15-9744-4915-9e2a-54678a2b0d97 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:30.341933 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f838f15-9744-4915-9e2a-54678a2b0d97 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:30.342437 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f838f15-9744-4915-9e2a-54678a2b0d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:30.377727 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-8f838f15-9744-4915-9e2a-54678a2b0d97 None None] MFA Rules not processed for user `8be4a2a8537749849260ca4b4ce7a4df`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:30.379349 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-8f838f15-9744-4915-9e2a-54678a2b0d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:30.386051 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-ebfd5d4e-10a1-4bc8-ba3a-12e7beeee800 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b81a0dbb56b4f9393b056686381105c', 'request_id': 'req-ebfd5d4e-10a1-4bc8-ba3a-12e7beeee800'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:30.392790 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-ebfd5d4e-10a1-4bc8-ba3a-12e7beeee800 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b81a0dbb56b4f9393b056686381105c', 'request_id': 'req-ebfd5d4e-10a1-4bc8-ba3a-12e7beeee800'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:30.400200 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-ebfd5d4e-10a1-4bc8-ba3a-12e7beeee800 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b81a0dbb56b4f9393b056686381105c', 'request_id': 'req-ebfd5d4e-10a1-4bc8-ba3a-12e7beeee800'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:30.415680 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-ebfd5d4e-10a1-4bc8-ba3a-12e7beeee800 admin admin] The token cache is being invalidate because project 1b81a0dbb56b4f9393b056686381105c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:29:30.416114 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-ebfd5d4e-10a1-4bc8-ba3a-12e7beeee800 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:30.416923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-ebfd5d4e-10a1-4bc8-ba3a-12e7beeee800 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:30.418792 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1067/2143] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:29:30 2022] DELETE /identity/v3/projects/1b81a0dbb56b4f9393b056686381105c => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:30.420231 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6e66c-5367-4c70-a646-daab9256edc8 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:30.420661 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6e66c-5367-4c70-a646-daab9256edc8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:30.420769 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6e66c-5367-4c70-a646-daab9256edc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:30.433942 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:30.439816 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-84b6e66c-5367-4c70-a646-daab9256edc8 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:30.441132 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-84b6e66c-5367-4c70-a646-daab9256edc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:30.454237 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1077/2144] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:30.458787 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209812b7-b41a-461c-aeff-16a5e098b23d None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:30.459244 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:30.459829 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209812b7-b41a-461c-aeff-16a5e098b23d None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:30.460258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209812b7-b41a-461c-aeff-16a5e098b23d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:30.481292 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-209812b7-b41a-461c-aeff-16a5e098b23d None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:30.482505 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-209812b7-b41a-461c-aeff-16a5e098b23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:30.503396 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1068/2145] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:30.504585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b6e66c-5367-4c70-a646-daab9256edc8 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:30.505860 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-84b6e66c-5367-4c70-a646-daab9256edc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:30.548049 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1078/2146] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:30.548976 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209812b7-b41a-461c-aeff-16a5e098b23d None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:30.560382 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209812b7-b41a-461c-aeff-16a5e098b23d None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:30.567937 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c76fa9-49f5-4145-ac9b-92c476df73d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:30.568725 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c76fa9-49f5-4145-ac9b-92c476df73d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:30.569147 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c76fa9-49f5-4145-ac9b-92c476df73d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:30.569612 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c76fa9-49f5-4145-ac9b-92c476df73d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:30.570403 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c76fa9-49f5-4145-ac9b-92c476df73d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:29:30.570845 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c76fa9-49f5-4145-ac9b-92c476df73d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:30.572585 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c76fa9-49f5-4145-ac9b-92c476df73d3 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:30.588998 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1079/2147] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:29:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:29:30.589197 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b6e66c-5367-4c70-a646-daab9256edc8 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:30.591016 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb62b2f-ccc6-4600-b830-8071f2fde377 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:30.591300 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb62b2f-ccc6-4600-b830-8071f2fde377 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:30.591568 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb62b2f-ccc6-4600-b830-8071f2fde377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:30.594911 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac82f2-3b0c-4155-b47f-1b9f10daebce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:30.595697 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac82f2-3b0c-4155-b47f-1b9f10daebce service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:30.596051 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac82f2-3b0c-4155-b47f-1b9f10daebce service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:30.596395 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac82f2-3b0c-4155-b47f-1b9f10daebce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:30.606487 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac82f2-3b0c-4155-b47f-1b9f10daebce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:30.608092 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac82f2-3b0c-4155-b47f-1b9f10daebce service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:30.611847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-ddb62b2f-ccc6-4600-b830-8071f2fde377 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:30.613101 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-ddb62b2f-ccc6-4600-b830-8071f2fde377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:30.617863 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1069/2148] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:29:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:29:30.618736 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac82f2-3b0c-4155-b47f-1b9f10daebce service nova] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:30.623958 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1080/2149] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:30.628695 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb62b2f-ccc6-4600-b830-8071f2fde377 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:30.632382 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac82f2-3b0c-4155-b47f-1b9f10daebce service nova] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:30.639727 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb62b2f-ccc6-4600-b830-8071f2fde377 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:30.642309 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88888a8e-1635-4e0c-a08f-6570c9754df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:30.643164 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88888a8e-1635-4e0c-a08f-6570c9754df5 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:30.643471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88888a8e-1635-4e0c-a08f-6570c9754df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:30.643829 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88888a8e-1635-4e0c-a08f-6570c9754df5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:30.643942 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6fd37f-42d2-49ad-b027-1868f96bdabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:30.644740 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6fd37f-42d2-49ad-b027-1868f96bdabd admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:30.644873 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88888a8e-1635-4e0c-a08f-6570c9754df5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:30.644982 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6fd37f-42d2-49ad-b027-1868f96bdabd admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:30.645131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6fd37f-42d2-49ad-b027-1868f96bdabd admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:30.649297 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6fd37f-42d2-49ad-b027-1868f96bdabd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:29:30.649297 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6fd37f-42d2-49ad-b027-1868f96bdabd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:30.649526 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88888a8e-1635-4e0c-a08f-6570c9754df5 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:30.649622 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6fd37f-42d2-49ad-b027-1868f96bdabd admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:30.676749 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1081/2150] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:29:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:29:30.680242 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6fd37f-42d2-49ad-b027-1868f96bdabd admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:30.697916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6fd37f-42d2-49ad-b027-1868f96bdabd admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:30.702117 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1070/2151] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:29:30 2022] POST /identity/v3/projects => generated 409 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:29:30.703273 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88888a8e-1635-4e0c-a08f-6570c9754df5 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:30.784231 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8faa9-0305-4c6c-acca-aceb4a5932ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:30.784733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8faa9-0305-4c6c-acca-aceb4a5932ed admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:30.784976 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8faa9-0305-4c6c-acca-aceb4a5932ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:30.785278 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8faa9-0305-4c6c-acca-aceb4a5932ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:30.785914 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c8faa9-0305-4c6c-acca-aceb4a5932ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:30.788159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c8faa9-0305-4c6c-acca-aceb4a5932ed admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:30.831533 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88888a8e-1635-4e0c-a08f-6570c9754df5 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:30.838243 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc869c-675d-4821-a8a2-6ca484459921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:30.839238 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc869c-675d-4821-a8a2-6ca484459921 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:30.840588 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc869c-675d-4821-a8a2-6ca484459921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:30.840944 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc869c-675d-4821-a8a2-6ca484459921 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:30.841672 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc869c-675d-4821-a8a2-6ca484459921 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:30.848786 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1082/2152] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:29:30 2022] POST /identity/v3/projects => generated 383 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:29:30.858211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8faa9-0305-4c6c-acca-aceb4a5932ed admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:30.861492 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc869c-675d-4821-a8a2-6ca484459921 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:30.893041 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1071/2153] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:29:30 2022] POST /identity/v3/users => generated 305 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:29:30.897513 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc869c-675d-4821-a8a2-6ca484459921 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:30.911467 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc869c-675d-4821-a8a2-6ca484459921 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:30.923390 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:30.927966 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:30.968387 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8faa9-0305-4c6c-acca-aceb4a5932ed admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:30.975520 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102613c8-af42-4fb5-876a-0ab3bfa570f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:30.976337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102613c8-af42-4fb5-876a-0ab3bfa570f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:30.976893 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102613c8-af42-4fb5-876a-0ab3bfa570f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:30.977168 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102613c8-af42-4fb5-876a-0ab3bfa570f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:30.977889 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102613c8-af42-4fb5-876a-0ab3bfa570f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:30.980226 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102613c8-af42-4fb5-876a-0ab3bfa570f6 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:31.006681 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3827672a-0928-4ba1-b29b-1e08715fd595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:31.007309 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827672a-0928-4ba1-b29b-1e08715fd595 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:31.007686 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1083/2154] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:29:30 2022] POST /identity/v3/users => generated 292 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:29:31.007821 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827672a-0928-4ba1-b29b-1e08715fd595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:31.007821 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827672a-0928-4ba1-b29b-1e08715fd595 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:31.008751 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102613c8-af42-4fb5-876a-0ab3bfa570f6 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:31.008871 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3827672a-0928-4ba1-b29b-1e08715fd595 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:31.010925 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3827672a-0928-4ba1-b29b-1e08715fd595 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:31.011057 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-102613c8-af42-4fb5-876a-0ab3bfa570f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:31.023390 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1072/2155] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:29:31.024292 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3827672a-0928-4ba1-b29b-1e08715fd595 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:31.027088 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102613c8-af42-4fb5-876a-0ab3bfa570f6 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:31.032024 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f191de2-f4a4-4b40-af67-6047b56d6b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:31.032745 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f191de2-f4a4-4b40-af67-6047b56d6b95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:31.033020 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f191de2-f4a4-4b40-af67-6047b56d6b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:31.033214 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f191de2-f4a4-4b40-af67-6047b56d6b95 admin admin] PATH_INFO: `/v3/users/8be4a2a8537749849260ca4b4ce7a4df` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:31.034766 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f191de2-f4a4-4b40-af67-6047b56d6b95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8be4a2a8537749849260ca4b4ce7a4df)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:31.035367 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3827672a-0928-4ba1-b29b-1e08715fd595 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:31.036701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f191de2-f4a4-4b40-af67-6047b56d6b95 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:31.040205 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e0b8a-3107-4430-82ed-6e75ae41ce59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:31.040881 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e0b8a-3107-4430-82ed-6e75ae41ce59 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:31.041231 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e0b8a-3107-4430-82ed-6e75ae41ce59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:31.041596 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e0b8a-3107-4430-82ed-6e75ae41ce59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:31.042241 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e0b8a-3107-4430-82ed-6e75ae41ce59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:31.044571 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e0b8a-3107-4430-82ed-6e75ae41ce59 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:31.054684 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1073/2156] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:29:31.058787 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e0b8a-3107-4430-82ed-6e75ae41ce59 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:31.068601 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e0b8a-3107-4430-82ed-6e75ae41ce59 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:31.074528 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1b3833-81bf-4d2c-bc7e-f558a300da52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:31.075139 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b3833-81bf-4d2c-bc7e-f558a300da52 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:31.075416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b3833-81bf-4d2c-bc7e-f558a300da52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:31.075685 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b3833-81bf-4d2c-bc7e-f558a300da52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:31.076317 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1b3833-81bf-4d2c-bc7e-f558a300da52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:31.077935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1b3833-81bf-4d2c-bc7e-f558a300da52 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:31.089194 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1074/2157] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:29:31.090173 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1b3833-81bf-4d2c-bc7e-f558a300da52 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:31.096827 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-5f191de2-f4a4-4b40-af67-6047b56d6b95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8be4a2a8537749849260ca4b4ce7a4df'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:31.100148 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1b3833-81bf-4d2c-bc7e-f558a300da52 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:31.106727 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291e90a-d4df-4325-9625-2ff6e0a9fc64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:31.107338 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291e90a-d4df-4325-9625-2ff6e0a9fc64 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:31.107617 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291e90a-d4df-4325-9625-2ff6e0a9fc64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:31.107886 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291e90a-d4df-4325-9625-2ff6e0a9fc64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:31.108649 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e291e90a-d4df-4325-9625-2ff6e0a9fc64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:31.110217 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e291e90a-d4df-4325-9625-2ff6e0a9fc64 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:31.111934 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-5f191de2-f4a4-4b40-af67-6047b56d6b95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8be4a2a8537749849260ca4b4ce7a4df'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:31.121601 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-5f191de2-f4a4-4b40-af67-6047b56d6b95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8be4a2a8537749849260ca4b4ce7a4df'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:31.124756 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1075/2158] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:29:31.125844 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291e90a-d4df-4325-9625-2ff6e0a9fc64 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:31.130810 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-5f191de2-f4a4-4b40-af67-6047b56d6b95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8be4a2a8537749849260ca4b4ce7a4df'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:31.131824 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-5f191de2-f4a4-4b40-af67-6047b56d6b95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8be4a2a8537749849260ca4b4ce7a4df'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:31.143699 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291e90a-d4df-4325-9625-2ff6e0a9fc64 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:31.148077 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1084/2159] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:29:31 2022] DELETE /identity/v3/users/8be4a2a8537749849260ca4b4ce7a4df => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:31.148841 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252adeed-ed36-415a-a38c-2e197caeee10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:31.149610 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252adeed-ed36-415a-a38c-2e197caeee10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:31.149896 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252adeed-ed36-415a-a38c-2e197caeee10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:31.150167 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252adeed-ed36-415a-a38c-2e197caeee10 admin admin] PATH_INFO: `/v3/projects/7e88f180d99f463ea172ed9704c9cafd/users/7a4f884ab7f94b4cade21a9a8ac78f85/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:31.150608 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f191de2-f4a4-4b40-af67-6047b56d6b95 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:31.152763 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-5f191de2-f4a4-4b40-af67-6047b56d6b95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:31.168952 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252adeed-ed36-415a-a38c-2e197caeee10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e88f180d99f463ea172ed9704c9cafd, user_id=7a4f884ab7f94b4cade21a9a8ac78f85, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:31.170483 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252adeed-ed36-415a-a38c-2e197caeee10 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:31.184699 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1076/2160] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:29:31 2022] PUT /identity/v3/projects/7e88f180d99f463ea172ed9704c9cafd/users/7a4f884ab7f94b4cade21a9a8ac78f85/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:31.186153 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420260ef-5fc6-41a4-8c97-bed9337df041 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:31.186559 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420260ef-5fc6-41a4-8c97-bed9337df041 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:31.186828 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420260ef-5fc6-41a4-8c97-bed9337df041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:31.213704 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-420260ef-5fc6-41a4-8c97-bed9337df041 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:31.215674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-420260ef-5fc6-41a4-8c97-bed9337df041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:31.249786 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f191de2-f4a4-4b40-af67-6047b56d6b95 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:31.298199 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1077/2161] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:31.299190 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:31.299569 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55135c12-ff4a-4b09-81d9-78667d19c705 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:31.299857 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55135c12-ff4a-4b09-81d9-78667d19c705 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:31.300138 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55135c12-ff4a-4b09-81d9-78667d19c705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:31.319980 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdeae41-4f47-4053-b5d8-25b59d7e45a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:31.320662 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdeae41-4f47-4053-b5d8-25b59d7e45a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:31.320962 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdeae41-4f47-4053-b5d8-25b59d7e45a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:31.321197 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdeae41-4f47-4053-b5d8-25b59d7e45a0 admin admin] PATH_INFO: `/v3/projects/61f9c257416f456db4f2e0cbdfdfa616/users/24b8f1c4cdf54c36afbe48134cd210d8/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:31.332963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-55135c12-ff4a-4b09-81d9-78667d19c705 None None] MFA Rules not processed for user `7a4f884ab7f94b4cade21a9a8ac78f85`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:31.334479 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-55135c12-ff4a-4b09-81d9-78667d19c705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:31.337982 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdeae41-4f47-4053-b5d8-25b59d7e45a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61f9c257416f456db4f2e0cbdfdfa616, user_id=24b8f1c4cdf54c36afbe48134cd210d8, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:31.341175 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdeae41-4f47-4053-b5d8-25b59d7e45a0 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:31.355571 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1085/2162] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:29:31 2022] PUT /identity/v3/projects/61f9c257416f456db4f2e0cbdfdfa616/users/24b8f1c4cdf54c36afbe48134cd210d8/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:31.356435 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdeae41-4f47-4053-b5d8-25b59d7e45a0 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:31.358153 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-fbdeae41-4f47-4053-b5d8-25b59d7e45a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:31.387765 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:31.412741 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1078/2163] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:31 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:31.416601 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e781fa-4c6f-44ed-9fae-c1370a4c02ba None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:31.416601 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e781fa-4c6f-44ed-9fae-c1370a4c02ba None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:31.416601 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e781fa-4c6f-44ed-9fae-c1370a4c02ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:31.422995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdeae41-4f47-4053-b5d8-25b59d7e45a0 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:31.432321 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c100fd91-537f-4487-9085-16ceb6506d96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:31.433996 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100fd91-537f-4487-9085-16ceb6506d96 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:31.434874 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100fd91-537f-4487-9085-16ceb6506d96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:31.435307 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100fd91-537f-4487-9085-16ceb6506d96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:31.450650 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-49e781fa-4c6f-44ed-9fae-c1370a4c02ba None None] MFA Rules not processed for user `24b8f1c4cdf54c36afbe48134cd210d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:31.452636 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-49e781fa-4c6f-44ed-9fae-c1370a4c02ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:31.520139 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c100fd91-537f-4487-9085-16ceb6506d96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:31.522034 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c100fd91-537f-4487-9085-16ceb6506d96 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:31.536084 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1086/2164] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:29:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:29:31.538310 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c100fd91-537f-4487-9085-16ceb6506d96 service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:31.539641 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-c100fd91-537f-4487-9085-16ceb6506d96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:31.552378 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c100fd91-537f-4487-9085-16ceb6506d96 service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:31.557643 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1079/2165] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:31.557774 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddab6c3c-082f-4244-93a3-83d2a38791cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:31.558761 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab6c3c-082f-4244-93a3-83d2a38791cc admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:31.559041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab6c3c-082f-4244-93a3-83d2a38791cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:31.559312 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab6c3c-082f-4244-93a3-83d2a38791cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:31.559944 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddab6c3c-082f-4244-93a3-83d2a38791cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:31.562250 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddab6c3c-082f-4244-93a3-83d2a38791cc admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:31.562581 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e781fa-4c6f-44ed-9fae-c1370a4c02ba None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:31.575685 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e781fa-4c6f-44ed-9fae-c1370a4c02ba None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:31.582606 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4eb5b-4fac-4ec4-b1a4-82346c941600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:31.584653 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4eb5b-4fac-4ec4-b1a4-82346c941600 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:31.585586 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4eb5b-4fac-4ec4-b1a4-82346c941600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:31.586357 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4eb5b-4fac-4ec4-b1a4-82346c941600 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:31.587081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b4eb5b-4fac-4ec4-b1a4-82346c941600 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:31.590319 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b4eb5b-4fac-4ec4-b1a4-82346c941600 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:31.595870 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1087/2166] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:29:31 2022] POST /identity/v3/projects => generated 405 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:29:31.597226 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddab6c3c-082f-4244-93a3-83d2a38791cc admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:31.634951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1080/2167] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:29:31 2022] POST /identity/v3/projects => generated 383 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:29:31.639065 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4eb5b-4fac-4ec4-b1a4-82346c941600 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:31.696076 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddab6c3c-082f-4244-93a3-83d2a38791cc admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:31.702343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:31.720511 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4eb5b-4fac-4ec4-b1a4-82346c941600 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:31.728947 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0f025-c671-4aea-8ba2-4b5278a68004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:31.729740 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0f025-c671-4aea-8ba2-4b5278a68004 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:31.730667 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0f025-c671-4aea-8ba2-4b5278a68004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:31.731003 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0f025-c671-4aea-8ba2-4b5278a68004 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:31.731783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0f025-c671-4aea-8ba2-4b5278a68004 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:31.733989 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0f025-c671-4aea-8ba2-4b5278a68004 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:31.760696 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1081/2168] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:29:31 2022] POST /identity/v3/users => generated 298 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:29:31.765229 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0f025-c671-4aea-8ba2-4b5278a68004 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:31.777530 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0f025-c671-4aea-8ba2-4b5278a68004 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:31.782594 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b47ccd-0f0c-4fdf-bd2b-44c0b8d1a6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:31.783153 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b47ccd-0f0c-4fdf-bd2b-44c0b8d1a6ac admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:31.783431 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b47ccd-0f0c-4fdf-bd2b-44c0b8d1a6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:31.783700 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b47ccd-0f0c-4fdf-bd2b-44c0b8d1a6ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:31.784345 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b47ccd-0f0c-4fdf-bd2b-44c0b8d1a6ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:31.785360 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6be503b-e675-49b6-8d9f-380ac16173b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:31.786053 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b47ccd-0f0c-4fdf-bd2b-44c0b8d1a6ac admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:31.786290 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be503b-e675-49b6-8d9f-380ac16173b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:31.786754 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be503b-e675-49b6-8d9f-380ac16173b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:31.787128 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be503b-e675-49b6-8d9f-380ac16173b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:31.788770 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6be503b-e675-49b6-8d9f-380ac16173b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:31.790829 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6be503b-e675-49b6-8d9f-380ac16173b6 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:31.797200 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1082/2169] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:29:31.801335 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b47ccd-0f0c-4fdf-bd2b-44c0b8d1a6ac admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:31.811877 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b47ccd-0f0c-4fdf-bd2b-44c0b8d1a6ac admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:31.816804 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff9ec2-29ae-4445-af49-4cebcee2d255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:31.816964 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1088/2170] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:29:31 2022] POST /identity/v3/users => generated 309 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:29:31.820110 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff9ec2-29ae-4445-af49-4cebcee2d255 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:31.820403 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff9ec2-29ae-4445-af49-4cebcee2d255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:31.820595 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6be503b-e675-49b6-8d9f-380ac16173b6 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:31.820713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff9ec2-29ae-4445-af49-4cebcee2d255 admin admin] PATH_INFO: `/v3/projects/20a2cfd8cde44efd9147d9a792de7341/users/1beac350ec7043b287acbfde6996d942/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:31.831956 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6be503b-e675-49b6-8d9f-380ac16173b6 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:31.837289 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d8a2a-2787-4ad6-b162-2aa3fa90f2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:31.837920 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d8a2a-2787-4ad6-b162-2aa3fa90f2fc admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:31.838029 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdff9ec2-29ae-4445-af49-4cebcee2d255 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20a2cfd8cde44efd9147d9a792de7341, user_id=1beac350ec7043b287acbfde6996d942, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:31.838615 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d8a2a-2787-4ad6-b162-2aa3fa90f2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:31.838615 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d8a2a-2787-4ad6-b162-2aa3fa90f2fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:31.839551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d8a2a-2787-4ad6-b162-2aa3fa90f2fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:31.841103 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdff9ec2-29ae-4445-af49-4cebcee2d255 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:31.841213 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d8a2a-2787-4ad6-b162-2aa3fa90f2fc admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:31.856791 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1089/2171] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:29:31.861288 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d8a2a-2787-4ad6-b162-2aa3fa90f2fc admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:31.861435 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1083/2172] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:29:31 2022] PUT /identity/v3/projects/20a2cfd8cde44efd9147d9a792de7341/users/1beac350ec7043b287acbfde6996d942/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:29:31.864568 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff9ec2-29ae-4445-af49-4cebcee2d255 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:31.872504 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d8a2a-2787-4ad6-b162-2aa3fa90f2fc admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:31.875149 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:31.932235 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff9ec2-29ae-4445-af49-4cebcee2d255 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:31.936140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7e020-0b04-4b73-ac75-44ebbee8c79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:31.936803 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7e020-0b04-4b73-ac75-44ebbee8c79b admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:31.936965 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ef6ed-0212-4447-9e6c-45ea34a1f2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:31.937102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7e020-0b04-4b73-ac75-44ebbee8c79b admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:31.937393 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7e020-0b04-4b73-ac75-44ebbee8c79b admin admin] PATH_INFO: `/v3/projects/cfb9ddecf33b40d38e79472c932914de/users/a6f7f6dd8d004e8da95cfe446c4161af/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:31.937619 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ef6ed-0212-4447-9e6c-45ea34a1f2ee admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:31.937898 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ef6ed-0212-4447-9e6c-45ea34a1f2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:31.938171 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ef6ed-0212-4447-9e6c-45ea34a1f2ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:31.938810 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ef6ed-0212-4447-9e6c-45ea34a1f2ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:31.940401 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ef6ed-0212-4447-9e6c-45ea34a1f2ee admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:31.951208 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1084/2173] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:29:31.952038 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ef6ed-0212-4447-9e6c-45ea34a1f2ee admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:31.955574 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b70ef6ed-0212-4447-9e6c-45ea34a1f2ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:31.957415 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb7e020-0b04-4b73-ac75-44ebbee8c79b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfb9ddecf33b40d38e79472c932914de, user_id=a6f7f6dd8d004e8da95cfe446c4161af, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:31.960546 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb7e020-0b04-4b73-ac75-44ebbee8c79b admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:31.969927 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ef6ed-0212-4447-9e6c-45ea34a1f2ee admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:31.975820 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1090/2174] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:29:31 2022] PUT /identity/v3/projects/cfb9ddecf33b40d38e79472c932914de/users/a6f7f6dd8d004e8da95cfe446c4161af/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:31.976615 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7e020-0b04-4b73-ac75-44ebbee8c79b admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:32.045665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f917b94-490c-4bb4-a2e3-3b542d243780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:32.046732 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f917b94-490c-4bb4-a2e3-3b542d243780 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:32.047237 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f917b94-490c-4bb4-a2e3-3b542d243780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:32.047727 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f917b94-490c-4bb4-a2e3-3b542d243780 admin admin] PATH_INFO: `/v3/projects/2e95b9f1ae2142b4b408191b19f1755c` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:32.049077 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f917b94-490c-4bb4-a2e3-3b542d243780 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e95b9f1ae2142b4b408191b19f1755c)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:32.053001 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f917b94-490c-4bb4-a2e3-3b542d243780 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:32.056787 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7e020-0b04-4b73-ac75-44ebbee8c79b admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:32.064064 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66caab2f-5bce-4678-9b29-904af22c8f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:32.065926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66caab2f-5bce-4678-9b29-904af22c8f66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:32.065926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66caab2f-5bce-4678-9b29-904af22c8f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:32.066195 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66caab2f-5bce-4678-9b29-904af22c8f66 admin admin] PATH_INFO: `/v3/projects/20a2cfd8cde44efd9147d9a792de7341/users/1beac350ec7043b287acbfde6996d942/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:32.068975 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66caab2f-5bce-4678-9b29-904af22c8f66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20a2cfd8cde44efd9147d9a792de7341, user_id=1beac350ec7043b287acbfde6996d942, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:32.072100 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66caab2f-5bce-4678-9b29-904af22c8f66 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:32.088305 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1091/2175] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:29:31 2022] PUT /identity/v3/projects/20a2cfd8cde44efd9147d9a792de7341/users/1beac350ec7043b287acbfde6996d942/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:32.089219 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66caab2f-5bce-4678-9b29-904af22c8f66 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:32.117766 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:32.137130 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0f917b94-490c-4bb4-a2e3-3b542d243780 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e95b9f1ae2142b4b408191b19f1755c', 'request_id': 'req-0f917b94-490c-4bb4-a2e3-3b542d243780'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:32.157444 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0f917b94-490c-4bb4-a2e3-3b542d243780 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e95b9f1ae2142b4b408191b19f1755c', 'request_id': 'req-0f917b94-490c-4bb4-a2e3-3b542d243780'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:32.168153 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0f917b94-490c-4bb4-a2e3-3b542d243780 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e95b9f1ae2142b4b408191b19f1755c', 'request_id': 'req-0f917b94-490c-4bb4-a2e3-3b542d243780'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:32.184296 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0f917b94-490c-4bb4-a2e3-3b542d243780 admin admin] The token cache is being invalidate because project 2e95b9f1ae2142b4b408191b19f1755c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:29:32.184934 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0f917b94-490c-4bb4-a2e3-3b542d243780 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:32.185414 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0f917b94-490c-4bb4-a2e3-3b542d243780 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:32.187223 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1085/2176] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:29:31 2022] DELETE /identity/v3/projects/2e95b9f1ae2142b4b408191b19f1755c => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:32.188234 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dddf8f8-a544-44c3-b61f-93222dd245d2 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:32.188512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dddf8f8-a544-44c3-b61f-93222dd245d2 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:32.188779 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dddf8f8-a544-44c3-b61f-93222dd245d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:32.216282 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66caab2f-5bce-4678-9b29-904af22c8f66 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:32.218190 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-6dddf8f8-a544-44c3-b61f-93222dd245d2 None None] MFA Rules not processed for user `1beac350ec7043b287acbfde6996d942`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:32.219505 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6dddf8f8-a544-44c3-b61f-93222dd245d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:32.242477 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:32.285684 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2851bd28-bb87-4514-acc6-5895bd73dc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:32.286467 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2851bd28-bb87-4514-acc6-5895bd73dc26 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:32.286753 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2851bd28-bb87-4514-acc6-5895bd73dc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:32.287553 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2851bd28-bb87-4514-acc6-5895bd73dc26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:32.288214 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2851bd28-bb87-4514-acc6-5895bd73dc26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:32.289904 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2851bd28-bb87-4514-acc6-5895bd73dc26 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:32.296362 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1086/2177] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:32 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:32.297721 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792554a4-2ed1-4ab4-a374-a6f8e7abdfbc None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:32.298005 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792554a4-2ed1-4ab4-a374-a6f8e7abdfbc None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:32.298284 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792554a4-2ed1-4ab4-a374-a6f8e7abdfbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:32.305495 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1092/2178] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:29:32.306345 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2851bd28-bb87-4514-acc6-5895bd73dc26 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:32.307979 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-2851bd28-bb87-4514-acc6-5895bd73dc26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:32.320251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2851bd28-bb87-4514-acc6-5895bd73dc26 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:32.320540 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-792554a4-2ed1-4ab4-a374-a6f8e7abdfbc None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:32.321760 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-792554a4-2ed1-4ab4-a374-a6f8e7abdfbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:32.324880 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f010f605-bd3c-495a-aa97-eb12d64c0b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:32.325562 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010f605-bd3c-495a-aa97-eb12d64c0b7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:32.325841 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010f605-bd3c-495a-aa97-eb12d64c0b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:32.326120 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010f605-bd3c-495a-aa97-eb12d64c0b7a admin admin] PATH_INFO: `/v3/projects/cfb9ddecf33b40d38e79472c932914de/users/a6f7f6dd8d004e8da95cfe446c4161af/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:32.327978 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f010f605-bd3c-495a-aa97-eb12d64c0b7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfb9ddecf33b40d38e79472c932914de, user_id=a6f7f6dd8d004e8da95cfe446c4161af, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:32.329639 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1087/2179] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:32.330647 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792554a4-2ed1-4ab4-a374-a6f8e7abdfbc None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:32.330886 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f010f605-bd3c-495a-aa97-eb12d64c0b7a admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:32.331714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-792554a4-2ed1-4ab4-a374-a6f8e7abdfbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:32.342748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1093/2180] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:29:32 2022] PUT /identity/v3/projects/cfb9ddecf33b40d38e79472c932914de/users/a6f7f6dd8d004e8da95cfe446c4161af/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:32.350257 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965def2-0567-4ba5-a2b6-449981753611 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:32.350541 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965def2-0567-4ba5-a2b6-449981753611 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:32.350888 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965def2-0567-4ba5-a2b6-449981753611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:32.371448 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-c965def2-0567-4ba5-a2b6-449981753611 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:32.372639 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-c965def2-0567-4ba5-a2b6-449981753611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:32.390098 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792554a4-2ed1-4ab4-a374-a6f8e7abdfbc None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:32.432576 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1094/2181] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:32.435855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754477be-b2d5-4b07-bad9-1cf8cb7039a6 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:32.436714 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754477be-b2d5-4b07-bad9-1cf8cb7039a6 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:32.437518 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754477be-b2d5-4b07-bad9-1cf8cb7039a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:32.439717 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c72864-de8f-4199-80dd-9f6d8ae1eb94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:32.440335 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c72864-de8f-4199-80dd-9f6d8ae1eb94 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:32.440614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c72864-de8f-4199-80dd-9f6d8ae1eb94 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:32.440891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c72864-de8f-4199-80dd-9f6d8ae1eb94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:32.443283 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-22c72864-de8f-4199-80dd-9f6d8ae1eb94 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:32.468600 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-754477be-b2d5-4b07-bad9-1cf8cb7039a6 None None] MFA Rules not processed for user `a6f7f6dd8d004e8da95cfe446c4161af`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:32.469849 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-754477be-b2d5-4b07-bad9-1cf8cb7039a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:32.496111 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:32.506208 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c72864-de8f-4199-80dd-9f6d8ae1eb94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:32.507646 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c72864-de8f-4199-80dd-9f6d8ae1eb94 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:32.517060 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1088/2182] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:29:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:29:32.517902 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c72864-de8f-4199-80dd-9f6d8ae1eb94 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:32.529001 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c72864-de8f-4199-80dd-9f6d8ae1eb94 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:32.533921 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d746975-eaed-4f21-90ff-844b1372ebec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:32.534573 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d746975-eaed-4f21-90ff-844b1372ebec admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:32.534921 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d746975-eaed-4f21-90ff-844b1372ebec admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:32.535597 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d746975-eaed-4f21-90ff-844b1372ebec admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:32.535954 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d746975-eaed-4f21-90ff-844b1372ebec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:29:32.536261 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d746975-eaed-4f21-90ff-844b1372ebec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:32.537877 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d746975-eaed-4f21-90ff-844b1372ebec admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:32.544963 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1095/2183] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:32 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:32.555732 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1089/2184] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:29:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:29:32.555878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754477be-b2d5-4b07-bad9-1cf8cb7039a6 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:32.556715 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d746975-eaed-4f21-90ff-844b1372ebec admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:32.557198 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-754477be-b2d5-4b07-bad9-1cf8cb7039a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:32.567886 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d746975-eaed-4f21-90ff-844b1372ebec admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:32.572293 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754477be-b2d5-4b07-bad9-1cf8cb7039a6 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:32.572423 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b2e62-be1a-4480-9fd8-db8b5d727501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:32.573074 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b2e62-be1a-4480-9fd8-db8b5d727501 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:32.573394 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b2e62-be1a-4480-9fd8-db8b5d727501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:32.574407 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b2e62-be1a-4480-9fd8-db8b5d727501 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:32.574982 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5b2e62-be1a-4480-9fd8-db8b5d727501 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:32.577750 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5b2e62-be1a-4480-9fd8-db8b5d727501 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:32.577933 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd6df3-5a74-4945-9a21-70cc9a386f58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:32.578718 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd6df3-5a74-4945-9a21-70cc9a386f58 service cinder] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:32.579086 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd6df3-5a74-4945-9a21-70cc9a386f58 service cinder] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:32.579536 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd6df3-5a74-4945-9a21-70cc9a386f58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:32.586769 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:32.594648 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd6df3-5a74-4945-9a21-70cc9a386f58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:32.596371 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd6df3-5a74-4945-9a21-70cc9a386f58 service cinder] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:32.613220 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1096/2185] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:29:32 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:29:32.627417 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4faac7-d815-49da-92e1-bdf0bb5dc46e None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:32.627712 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4faac7-d815-49da-92e1-bdf0bb5dc46e None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:32.627992 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4faac7-d815-49da-92e1-bdf0bb5dc46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:32.628711 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1090/2186] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:29:32 2022] POST /identity/v3/projects => generated 355 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:29:32.633478 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b2e62-be1a-4480-9fd8-db8b5d727501 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:32.654292 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-2a4faac7-d815-49da-92e1-bdf0bb5dc46e None None] MFA Rules not processed for user `892bbd02129b417c987f4d429e21a833`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:32.655512 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-2a4faac7-d815-49da-92e1-bdf0bb5dc46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:32.727320 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b2e62-be1a-4480-9fd8-db8b5d727501 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:32.733021 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0701031-606a-41e9-864c-88942054207b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:32.733691 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0701031-606a-41e9-864c-88942054207b admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:32.735964 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0701031-606a-41e9-864c-88942054207b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:32.735964 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0701031-606a-41e9-864c-88942054207b admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:32.735964 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0701031-606a-41e9-864c-88942054207b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:32.736801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0701031-606a-41e9-864c-88942054207b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:32.754994 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1097/2187] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:32.769997 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1091/2188] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:29:32 2022] POST /identity/v3/users => generated 278 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:29:32.772021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4faac7-d815-49da-92e1-bdf0bb5dc46e None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:32.791958 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4faac7-d815-49da-92e1-bdf0bb5dc46e None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:32.798334 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded9de7-42b1-4cf3-b6ba-c0f0eea2f309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:32.798994 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded9de7-42b1-4cf3-b6ba-c0f0eea2f309 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:32.799275 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded9de7-42b1-4cf3-b6ba-c0f0eea2f309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:32.799789 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded9de7-42b1-4cf3-b6ba-c0f0eea2f309 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:32.800322 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded9de7-42b1-4cf3-b6ba-c0f0eea2f309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:32.801985 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded9de7-42b1-4cf3-b6ba-c0f0eea2f309 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:32.817905 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1098/2189] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:29:32.819208 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0701031-606a-41e9-864c-88942054207b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:32.825779 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded9de7-42b1-4cf3-b6ba-c0f0eea2f309 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:32.835496 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0701031-606a-41e9-864c-88942054207b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:32.844314 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68979e63-e344-4f2c-b498-47a689da73cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:32.846114 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68979e63-e344-4f2c-b498-47a689da73cc admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:32.846629 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68979e63-e344-4f2c-b498-47a689da73cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:32.847057 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68979e63-e344-4f2c-b498-47a689da73cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:32.848684 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68979e63-e344-4f2c-b498-47a689da73cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:32.851581 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68979e63-e344-4f2c-b498-47a689da73cc admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:32.863999 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1092/2190] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:29:32.868595 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68979e63-e344-4f2c-b498-47a689da73cc admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:32.889749 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68979e63-e344-4f2c-b498-47a689da73cc admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:32.896703 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e91cff-f705-4535-b0bc-c59cde27a2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:32.897153 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e91cff-f705-4535-b0bc-c59cde27a2e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:32.897471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e91cff-f705-4535-b0bc-c59cde27a2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:32.897752 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e91cff-f705-4535-b0bc-c59cde27a2e8 admin admin] PATH_INFO: `/v3/projects/9b9d6b789ba942ef8afd5821269a2782/users/dac1bb313ddf435181219337588777cc/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:32.901473 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded9de7-42b1-4cf3-b6ba-c0f0eea2f309 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:32.906048 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83786588-11b3-454b-8a87-602e0300ed6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:32.906912 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83786588-11b3-454b-8a87-602e0300ed6b service cinder] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:32.907337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83786588-11b3-454b-8a87-602e0300ed6b service cinder] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:32.907755 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83786588-11b3-454b-8a87-602e0300ed6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:32.909901 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-83786588-11b3-454b-8a87-602e0300ed6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:32.917962 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e91cff-f705-4535-b0bc-c59cde27a2e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b9d6b789ba942ef8afd5821269a2782, user_id=dac1bb313ddf435181219337588777cc, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:32.922459 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e91cff-f705-4535-b0bc-c59cde27a2e8 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:32.941648 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1093/2191] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:29:32 2022] PUT /identity/v3/projects/9b9d6b789ba942ef8afd5821269a2782/users/dac1bb313ddf435181219337588777cc/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:32.957631 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f2f4a-3075-4c96-83e3-833d88b57ff6 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:32.958472 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f2f4a-3075-4c96-83e3-833d88b57ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:32.959475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f2f4a-3075-4c96-83e3-833d88b57ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:32.994624 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:32.997955 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83786588-11b3-454b-8a87-602e0300ed6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:33.000570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83786588-11b3-454b-8a87-602e0300ed6b service cinder] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:33.003789 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-3f9f2f4a-3075-4c96-83e3-833d88b57ff6 None None] MFA Rules not processed for user `dac1bb313ddf435181219337588777cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:33.006400 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3f9f2f4a-3075-4c96-83e3-833d88b57ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:33.015893 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1099/2192] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:29:32 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 20:29:33.099556 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1094/2193] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:32 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:29:33.101384 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83786588-11b3-454b-8a87-602e0300ed6b service cinder] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:33.118077 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83786588-11b3-454b-8a87-602e0300ed6b service cinder] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:33.186504 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b7094-b6fd-45a4-8436-40fa384850e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:33.187122 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b7094-b6fd-45a4-8436-40fa384850e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:33.187404 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b7094-b6fd-45a4-8436-40fa384850e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:33.187676 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b7094-b6fd-45a4-8436-40fa384850e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:33.188884 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b7094-b6fd-45a4-8436-40fa384850e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:33.192262 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b7094-b6fd-45a4-8436-40fa384850e7 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:33.223467 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1100/2194] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:29:33 2022] POST /identity/v3/projects => generated 355 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:29:33.227450 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f2f4a-3075-4c96-83e3-833d88b57ff6 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:33.243038 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f2f4a-3075-4c96-83e3-833d88b57ff6 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:33.278394 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:33.324173 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e8d5c-c25d-499b-8b2b-d6fe6452da92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:33.324804 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e8d5c-c25d-499b-8b2b-d6fe6452da92 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:33.325090 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e8d5c-c25d-499b-8b2b-d6fe6452da92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:33.325506 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e8d5c-c25d-499b-8b2b-d6fe6452da92 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:33.326072 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9e8d5c-c25d-499b-8b2b-d6fe6452da92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:33.338756 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9e8d5c-c25d-499b-8b2b-d6fe6452da92 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:33.383531 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1095/2195] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:29:33 2022] POST /identity/v3/users => generated 284 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:29:33.388239 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b7094-b6fd-45a4-8436-40fa384850e7 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:33.399084 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b7094-b6fd-45a4-8436-40fa384850e7 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:33.411396 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e4e30-707a-41cc-bc58-865b5368b572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:33.412437 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e4e30-707a-41cc-bc58-865b5368b572 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:33.412891 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e4e30-707a-41cc-bc58-865b5368b572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:33.413174 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e4e30-707a-41cc-bc58-865b5368b572 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:33.413961 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e4e30-707a-41cc-bc58-865b5368b572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:33.415669 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e4e30-707a-41cc-bc58-865b5368b572 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:33.433780 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e8d5c-c25d-499b-8b2b-d6fe6452da92 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:33.456517 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1101/2196] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:29:33.461646 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e8d5c-c25d-499b-8b2b-d6fe6452da92 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:33.468783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d0cce-9a5a-4956-b160-06b29af7b69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:33.469566 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d0cce-9a5a-4956-b160-06b29af7b69d admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:33.470626 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d0cce-9a5a-4956-b160-06b29af7b69d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:33.470626 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d0cce-9a5a-4956-b160-06b29af7b69d admin admin] PATH_INFO: `/v3/projects/6d518257591540da9f6d1ba2ea5df4d2/users/fa3733fe2760466d945b82c2fad415ec/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:33.493062 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409d0cce-9a5a-4956-b160-06b29af7b69d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d518257591540da9f6d1ba2ea5df4d2, user_id=fa3733fe2760466d945b82c2fad415ec, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:33.496611 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409d0cce-9a5a-4956-b160-06b29af7b69d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:33.524416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1096/2197] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:29:33 2022] PUT /identity/v3/projects/6d518257591540da9f6d1ba2ea5df4d2/users/fa3733fe2760466d945b82c2fad415ec/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:33.529080 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e4e30-707a-41cc-bc58-865b5368b572 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:33.539337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e4e30-707a-41cc-bc58-865b5368b572 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:33.599420 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687ca6c-b593-4d41-bf6d-14087d03965e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:33.600039 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687ca6c-b593-4d41-bf6d-14087d03965e admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:33.600325 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687ca6c-b593-4d41-bf6d-14087d03965e admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:33.600605 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687ca6c-b593-4d41-bf6d-14087d03965e admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:33.601274 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687ca6c-b593-4d41-bf6d-14087d03965e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:33.602925 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687ca6c-b593-4d41-bf6d-14087d03965e admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:33.613077 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1102/2198] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:29:33.617611 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d0cce-9a5a-4956-b160-06b29af7b69d admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:33.629120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d0cce-9a5a-4956-b160-06b29af7b69d admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:33.633967 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30fd5a-9e4a-4aed-af50-02c1a254e6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:33.634561 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30fd5a-9e4a-4aed-af50-02c1a254e6c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:33.634837 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30fd5a-9e4a-4aed-af50-02c1a254e6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:33.635109 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30fd5a-9e4a-4aed-af50-02c1a254e6c6 admin admin] PATH_INFO: `/v3/projects/6d518257591540da9f6d1ba2ea5df4d2/users/fa3733fe2760466d945b82c2fad415ec/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:33.637650 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f30fd5a-9e4a-4aed-af50-02c1a254e6c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d518257591540da9f6d1ba2ea5df4d2, user_id=fa3733fe2760466d945b82c2fad415ec, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:33.640236 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f30fd5a-9e4a-4aed-af50-02c1a254e6c6 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:33.651523 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1097/2199] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:29:33 2022] PUT /identity/v3/projects/6d518257591540da9f6d1ba2ea5df4d2/users/fa3733fe2760466d945b82c2fad415ec/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:33.661801 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179d7a4-6f6b-43a0-9389-4002005a61f8 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:33.662092 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179d7a4-6f6b-43a0-9389-4002005a61f8 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:33.662365 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179d7a4-6f6b-43a0-9389-4002005a61f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:33.699460 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-3179d7a4-6f6b-43a0-9389-4002005a61f8 None None] MFA Rules not processed for user `fa3733fe2760466d945b82c2fad415ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:33.700666 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-3179d7a4-6f6b-43a0-9389-4002005a61f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:33.758400 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:33.780711 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1103/2200] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:33 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:33.794332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30fd5a-9e4a-4aed-af50-02c1a254e6c6 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:33.852407 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30fd5a-9e4a-4aed-af50-02c1a254e6c6 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:33.856351 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8796a0-b1c9-42b5-8a28-21e26c2d5d2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:33.857151 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8796a0-b1c9-42b5-8a28-21e26c2d5d2c service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:33.857618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8796a0-b1c9-42b5-8a28-21e26c2d5d2c service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:33.857825 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8796a0-b1c9-42b5-8a28-21e26c2d5d2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:33.868159 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8796a0-b1c9-42b5-8a28-21e26c2d5d2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:33.869839 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8796a0-b1c9-42b5-8a28-21e26c2d5d2c service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:33.880243 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1098/2201] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:29:33 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:29:33.955161 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3179d7a4-6f6b-43a0-9389-4002005a61f8 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:33.964712 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3179d7a4-6f6b-43a0-9389-4002005a61f8 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:33.968304 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996104d-790a-453d-9a2d-af220fff85b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:33.968892 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996104d-790a-453d-9a2d-af220fff85b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:33.969165 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996104d-790a-453d-9a2d-af220fff85b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:33.969464 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996104d-790a-453d-9a2d-af220fff85b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:34.024267 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9996104d-790a-453d-9a2d-af220fff85b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:34.025739 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9996104d-790a-453d-9a2d-af220fff85b7 service cinder] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:34.034402 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1104/2202] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:29:33 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:29:39.159107 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8796a0-b1c9-42b5-8a28-21e26c2d5d2c service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:39.169289 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8796a0-b1c9-42b5-8a28-21e26c2d5d2c service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:39.173397 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3a7e9-8740-4657-9b1e-6c28c7aacd6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:39.174011 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3a7e9-8740-4657-9b1e-6c28c7aacd6f service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:39.174292 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3a7e9-8740-4657-9b1e-6c28c7aacd6f service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:39.174723 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3a7e9-8740-4657-9b1e-6c28c7aacd6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:39.270843 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b3a7e9-8740-4657-9b1e-6c28c7aacd6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:39.270843 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b3a7e9-8740-4657-9b1e-6c28c7aacd6f service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:39.291388 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1099/2203] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:29:39 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:29:42.209551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996104d-790a-453d-9a2d-af220fff85b7 service cinder] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:42.341453 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996104d-790a-453d-9a2d-af220fff85b7 service cinder] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:42.349335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8761fe6e-db28-4e37-8fcc-1d3b0f9b13d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:42.350304 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761fe6e-db28-4e37-8fcc-1d3b0f9b13d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:42.350687 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761fe6e-db28-4e37-8fcc-1d3b0f9b13d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:42.351068 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761fe6e-db28-4e37-8fcc-1d3b0f9b13d0 admin admin] PATH_INFO: `/v3/users/7a4f884ab7f94b4cade21a9a8ac78f85` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:42.361226 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8761fe6e-db28-4e37-8fcc-1d3b0f9b13d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a4f884ab7f94b4cade21a9a8ac78f85)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:42.363456 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8761fe6e-db28-4e37-8fcc-1d3b0f9b13d0 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:42.423913 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-8761fe6e-db28-4e37-8fcc-1d3b0f9b13d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a4f884ab7f94b4cade21a9a8ac78f85'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:42.452159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-8761fe6e-db28-4e37-8fcc-1d3b0f9b13d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a4f884ab7f94b4cade21a9a8ac78f85'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:42.460041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-8761fe6e-db28-4e37-8fcc-1d3b0f9b13d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a4f884ab7f94b4cade21a9a8ac78f85'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:42.470415 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-8761fe6e-db28-4e37-8fcc-1d3b0f9b13d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a4f884ab7f94b4cade21a9a8ac78f85'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:42.472628 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-8761fe6e-db28-4e37-8fcc-1d3b0f9b13d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a4f884ab7f94b4cade21a9a8ac78f85'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:42.481048 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1105/2204] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:29:42 2022] DELETE /identity/v3/users/7a4f884ab7f94b4cade21a9a8ac78f85 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:42.485315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1ca70-315f-43b3-9b11-fb9cfffb44bb None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:42.486146 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1ca70-315f-43b3-9b11-fb9cfffb44bb None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:42.486575 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1ca70-315f-43b3-9b11-fb9cfffb44bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:42.508709 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-c8a1ca70-315f-43b3-9b11-fb9cfffb44bb None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:42.510370 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c8a1ca70-315f-43b3-9b11-fb9cfffb44bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:42.568164 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:42.639542 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1100/2205] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:42.647416 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8761fe6e-db28-4e37-8fcc-1d3b0f9b13d0 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:42.649123 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-8761fe6e-db28-4e37-8fcc-1d3b0f9b13d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:42.663704 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:42.737653 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8761fe6e-db28-4e37-8fcc-1d3b0f9b13d0 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:42.741140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d8dad-46e2-4f77-88ae-3c235a5086bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:42.741902 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d8dad-46e2-4f77-88ae-3c235a5086bc service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:42.742250 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d8dad-46e2-4f77-88ae-3c235a5086bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:42.742589 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d8dad-46e2-4f77-88ae-3c235a5086bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:42.752689 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158d8dad-46e2-4f77-88ae-3c235a5086bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:42.754244 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158d8dad-46e2-4f77-88ae-3c235a5086bc service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:42.764707 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1106/2206] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:29:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 20:29:43.149118 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a1ca70-315f-43b3-9b11-fb9cfffb44bb None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:43.150662 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c8a1ca70-315f-43b3-9b11-fb9cfffb44bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:43.163052 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a1ca70-315f-43b3-9b11-fb9cfffb44bb None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:43.168545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc712a-53d1-496f-9d3c-be7e6ca494fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:43.169181 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc712a-53d1-496f-9d3c-be7e6ca494fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:43.169529 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc712a-53d1-496f-9d3c-be7e6ca494fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:43.169811 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc712a-53d1-496f-9d3c-be7e6ca494fc admin admin] PATH_INFO: `/v3/projects/7e88f180d99f463ea172ed9704c9cafd` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:43.170924 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fc712a-53d1-496f-9d3c-be7e6ca494fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e88f180d99f463ea172ed9704c9cafd)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:43.172812 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fc712a-53d1-496f-9d3c-be7e6ca494fc admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:43.218027 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-29fc712a-53d1-496f-9d3c-be7e6ca494fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e88f180d99f463ea172ed9704c9cafd', 'request_id': 'req-29fc712a-53d1-496f-9d3c-be7e6ca494fc'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:43.223469 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-29fc712a-53d1-496f-9d3c-be7e6ca494fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e88f180d99f463ea172ed9704c9cafd', 'request_id': 'req-29fc712a-53d1-496f-9d3c-be7e6ca494fc'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:43.230163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-29fc712a-53d1-496f-9d3c-be7e6ca494fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e88f180d99f463ea172ed9704c9cafd', 'request_id': 'req-29fc712a-53d1-496f-9d3c-be7e6ca494fc'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:43.242062 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-29fc712a-53d1-496f-9d3c-be7e6ca494fc admin admin] The token cache is being invalidate because project 7e88f180d99f463ea172ed9704c9cafd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:29:43.242348 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-29fc712a-53d1-496f-9d3c-be7e6ca494fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:43.242958 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-29fc712a-53d1-496f-9d3c-be7e6ca494fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:43.244443 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1101/2207] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:29:43 2022] DELETE /identity/v3/projects/7e88f180d99f463ea172ed9704c9cafd => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:43.249008 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d8dad-46e2-4f77-88ae-3c235a5086bc service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:43.258394 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d8dad-46e2-4f77-88ae-3c235a5086bc service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:43.312807 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddce356-a213-4790-9b3f-d0bab61a74bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:43.313387 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddce356-a213-4790-9b3f-d0bab61a74bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:43.313665 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddce356-a213-4790-9b3f-d0bab61a74bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:43.313932 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddce356-a213-4790-9b3f-d0bab61a74bb admin admin] PATH_INFO: `/v3/users/a6f7f6dd8d004e8da95cfe446c4161af` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:43.315021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddce356-a213-4790-9b3f-d0bab61a74bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6f7f6dd8d004e8da95cfe446c4161af)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:43.316926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddce356-a213-4790-9b3f-d0bab61a74bb admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:43.362462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-fddce356-a213-4790-9b3f-d0bab61a74bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6f7f6dd8d004e8da95cfe446c4161af'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:43.375033 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-fddce356-a213-4790-9b3f-d0bab61a74bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6f7f6dd8d004e8da95cfe446c4161af'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:43.383216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-fddce356-a213-4790-9b3f-d0bab61a74bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6f7f6dd8d004e8da95cfe446c4161af'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:43.393235 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-fddce356-a213-4790-9b3f-d0bab61a74bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6f7f6dd8d004e8da95cfe446c4161af'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:43.393864 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-fddce356-a213-4790-9b3f-d0bab61a74bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6f7f6dd8d004e8da95cfe446c4161af'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:43.401779 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1107/2208] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:29:43 2022] DELETE /identity/v3/users/a6f7f6dd8d004e8da95cfe446c4161af => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:43.683487 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc712a-53d1-496f-9d3c-be7e6ca494fc admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:43.685707 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-29fc712a-53d1-496f-9d3c-be7e6ca494fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:43.699477 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc712a-53d1-496f-9d3c-be7e6ca494fc admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:43.757217 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3f0ba-49f7-40f8-80e6-eeb08a1388cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:43.757886 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3f0ba-49f7-40f8-80e6-eeb08a1388cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:43.758166 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3f0ba-49f7-40f8-80e6-eeb08a1388cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:43.758439 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3f0ba-49f7-40f8-80e6-eeb08a1388cb admin admin] PATH_INFO: `/v3/projects/cfb9ddecf33b40d38e79472c932914de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:43.759564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f3f0ba-49f7-40f8-80e6-eeb08a1388cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfb9ddecf33b40d38e79472c932914de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:43.762340 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f3f0ba-49f7-40f8-80e6-eeb08a1388cb admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:43.808000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:43.811704 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b4f3f0ba-49f7-40f8-80e6-eeb08a1388cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfb9ddecf33b40d38e79472c932914de', 'request_id': 'req-b4f3f0ba-49f7-40f8-80e6-eeb08a1388cb'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:43.819559 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b4f3f0ba-49f7-40f8-80e6-eeb08a1388cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfb9ddecf33b40d38e79472c932914de', 'request_id': 'req-b4f3f0ba-49f7-40f8-80e6-eeb08a1388cb'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:43.826831 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b4f3f0ba-49f7-40f8-80e6-eeb08a1388cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfb9ddecf33b40d38e79472c932914de', 'request_id': 'req-b4f3f0ba-49f7-40f8-80e6-eeb08a1388cb'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:43.842467 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b4f3f0ba-49f7-40f8-80e6-eeb08a1388cb admin admin] The token cache is being invalidate because project cfb9ddecf33b40d38e79472c932914de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:29:43.842775 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b4f3f0ba-49f7-40f8-80e6-eeb08a1388cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:43.843373 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b4f3f0ba-49f7-40f8-80e6-eeb08a1388cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:43.845423 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1102/2209] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:29:43 2022] DELETE /identity/v3/projects/cfb9ddecf33b40d38e79472c932914de => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:43.850100 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d0912-86ad-432b-b672-f8b8dcf178a3 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:43.850441 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d0912-86ad-432b-b672-f8b8dcf178a3 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:43.850659 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d0912-86ad-432b-b672-f8b8dcf178a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:43.873954 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-7e5d0912-86ad-432b-b672-f8b8dcf178a3 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:43.875204 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7e5d0912-86ad-432b-b672-f8b8dcf178a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:43.902481 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:43.942375 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1108/2210] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:43.950842 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f4299-6527-4aa3-9f2f-b9df9608cc06 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:43.951126 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f4299-6527-4aa3-9f2f-b9df9608cc06 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:43.951427 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f4299-6527-4aa3-9f2f-b9df9608cc06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:43.976776 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-407f4299-6527-4aa3-9f2f-b9df9608cc06 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:43.978026 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-407f4299-6527-4aa3-9f2f-b9df9608cc06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:43.990799 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1103/2211] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:43.996028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d0912-86ad-432b-b672-f8b8dcf178a3 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:44.006995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d0912-86ad-432b-b672-f8b8dcf178a3 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:44.012667 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f51a3ec-b124-43dd-abd1-3b397498a5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:44.013324 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51a3ec-b124-43dd-abd1-3b397498a5a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:44.013610 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51a3ec-b124-43dd-abd1-3b397498a5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:44.013882 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51a3ec-b124-43dd-abd1-3b397498a5a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:44.014528 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f51a3ec-b124-43dd-abd1-3b397498a5a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:29:44.014827 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f51a3ec-b124-43dd-abd1-3b397498a5a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:44.016973 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f51a3ec-b124-43dd-abd1-3b397498a5a8 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:44.032599 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1109/2212] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:29:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:29:44.033881 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f4299-6527-4aa3-9f2f-b9df9608cc06 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:44.048071 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f4299-6527-4aa3-9f2f-b9df9608cc06 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:44.053373 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e0338-dcc9-4f1c-a462-528133821c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:44.054126 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e0338-dcc9-4f1c-a462-528133821c62 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:44.054437 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e0338-dcc9-4f1c-a462-528133821c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:44.054717 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e0338-dcc9-4f1c-a462-528133821c62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:44.065818 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e0338-dcc9-4f1c-a462-528133821c62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:44.067736 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e0338-dcc9-4f1c-a462-528133821c62 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:44.109173 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1104/2213] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:29:44 2022] POST /identity/v3/projects => generated 383 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:29:44.113500 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f51a3ec-b124-43dd-abd1-3b397498a5a8 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:44.132243 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f51a3ec-b124-43dd-abd1-3b397498a5a8 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:44.199871 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f5b193-cce8-4939-b5ae-1ac006975f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:44.200439 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5b193-cce8-4939-b5ae-1ac006975f41 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:44.200720 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5b193-cce8-4939-b5ae-1ac006975f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:44.200992 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5b193-cce8-4939-b5ae-1ac006975f41 admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:44.201655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f5b193-cce8-4939-b5ae-1ac006975f41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:44.203573 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f5b193-cce8-4939-b5ae-1ac006975f41 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:44.226865 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1110/2214] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:29:44 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:29:44.231661 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e0338-dcc9-4f1c-a462-528133821c62 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:44.242685 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e0338-dcc9-4f1c-a462-528133821c62 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:44.248210 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f567f-7f80-4f42-8645-51363fb8fa88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:44.248979 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f567f-7f80-4f42-8645-51363fb8fa88 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:44.249289 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f567f-7f80-4f42-8645-51363fb8fa88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:44.249803 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f567f-7f80-4f42-8645-51363fb8fa88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:44.250455 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82f567f-7f80-4f42-8645-51363fb8fa88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:44.252679 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82f567f-7f80-4f42-8645-51363fb8fa88 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:44.271511 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1105/2215] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:29:44.272926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f5b193-cce8-4939-b5ae-1ac006975f41 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:44.284302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f5b193-cce8-4939-b5ae-1ac006975f41 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:44.289198 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a6213-1316-481e-b20f-a24c2611965c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:44.290029 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a6213-1316-481e-b20f-a24c2611965c admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:44.290477 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a6213-1316-481e-b20f-a24c2611965c admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:44.290978 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a6213-1316-481e-b20f-a24c2611965c admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:44.291722 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7a6213-1316-481e-b20f-a24c2611965c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:44.293511 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7a6213-1316-481e-b20f-a24c2611965c admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:44.304218 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1111/2216] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:29:44.308984 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f567f-7f80-4f42-8645-51363fb8fa88 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:44.319020 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f567f-7f80-4f42-8645-51363fb8fa88 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:44.323665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3778ecf-edd7-4fb5-8fae-b7679cdc196e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:44.324380 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3778ecf-edd7-4fb5-8fae-b7679cdc196e admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:44.324727 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3778ecf-edd7-4fb5-8fae-b7679cdc196e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:44.325072 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3778ecf-edd7-4fb5-8fae-b7679cdc196e admin admin] PATH_INFO: `/v3/projects/65c0a823d2db4790be0da2553b43a92c/users/1a37cb0446c842ebaedebdf1ec08e265/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:44.340649 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3778ecf-edd7-4fb5-8fae-b7679cdc196e admin admin] RBAC: Authorizing `identity:create_grant(project_id=65c0a823d2db4790be0da2553b43a92c, user_id=1a37cb0446c842ebaedebdf1ec08e265, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:44.343253 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3778ecf-edd7-4fb5-8fae-b7679cdc196e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:44.357520 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1106/2217] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:29:44 2022] PUT /identity/v3/projects/65c0a823d2db4790be0da2553b43a92c/users/1a37cb0446c842ebaedebdf1ec08e265/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:29:44.369403 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6741bfc-6df6-4ffe-b858-6f9bec41bfbe None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:44.369697 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6741bfc-6df6-4ffe-b858-6f9bec41bfbe None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:44.370515 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6741bfc-6df6-4ffe-b858-6f9bec41bfbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:44.404900 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-a6741bfc-6df6-4ffe-b858-6f9bec41bfbe None None] MFA Rules not processed for user `1a37cb0446c842ebaedebdf1ec08e265`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:44.406157 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a6741bfc-6df6-4ffe-b858-6f9bec41bfbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:44.487170 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1112/2218] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:44.492072 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3778ecf-edd7-4fb5-8fae-b7679cdc196e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:44.531845 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6741bfc-6df6-4ffe-b858-6f9bec41bfbe None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:44.535141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a6741bfc-6df6-4ffe-b858-6f9bec41bfbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:44.573895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3778ecf-edd7-4fb5-8fae-b7679cdc196e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:44.577028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:44.580932 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e152b42b-30c4-4e55-b0cb-7d8429df9cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:44.581910 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152b42b-30c4-4e55-b0cb-7d8429df9cb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:44.582227 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152b42b-30c4-4e55-b0cb-7d8429df9cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:44.582621 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152b42b-30c4-4e55-b0cb-7d8429df9cb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:44.583635 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e152b42b-30c4-4e55-b0cb-7d8429df9cb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:44.586316 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e152b42b-30c4-4e55-b0cb-7d8429df9cb1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:44.628576 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1107/2219] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:29:44 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:29:44.629900 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e152b42b-30c4-4e55-b0cb-7d8429df9cb1 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:44.632629 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6741bfc-6df6-4ffe-b858-6f9bec41bfbe None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:44.704863 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbed27e-0063-40e6-81cb-a21ae31c15d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:44.705531 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbed27e-0063-40e6-81cb-a21ae31c15d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:44.705812 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbed27e-0063-40e6-81cb-a21ae31c15d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:44.706088 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbed27e-0063-40e6-81cb-a21ae31c15d6 admin admin] PATH_INFO: `/v3/users/24b8f1c4cdf54c36afbe48134cd210d8` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:44.707212 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbed27e-0063-40e6-81cb-a21ae31c15d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24b8f1c4cdf54c36afbe48134cd210d8)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:44.711341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbed27e-0063-40e6-81cb-a21ae31c15d6 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:44.715739 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:44.731783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e152b42b-30c4-4e55-b0cb-7d8429df9cb1 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:44.738005 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5e63c-264c-444e-a7c1-7d5b9bf0b917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:44.738569 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5e63c-264c-444e-a7c1-7d5b9bf0b917 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:44.738983 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5e63c-264c-444e-a7c1-7d5b9bf0b917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:44.739123 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5e63c-264c-444e-a7c1-7d5b9bf0b917 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:44.740442 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a5e63c-264c-444e-a7c1-7d5b9bf0b917 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:44.743544 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a5e63c-264c-444e-a7c1-7d5b9bf0b917 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:44.782629 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1108/2220] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:29:44 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:29:44.787634 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-afbed27e-0063-40e6-81cb-a21ae31c15d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24b8f1c4cdf54c36afbe48134cd210d8'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:44.787940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5e63c-264c-444e-a7c1-7d5b9bf0b917 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:44.817828 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5e63c-264c-444e-a7c1-7d5b9bf0b917 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:44.818036 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-afbed27e-0063-40e6-81cb-a21ae31c15d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24b8f1c4cdf54c36afbe48134cd210d8'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:44.824258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942bcd7-c6b1-4564-a62d-1e29ceccdc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:44.825035 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942bcd7-c6b1-4564-a62d-1e29ceccdc8b admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:44.826448 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-afbed27e-0063-40e6-81cb-a21ae31c15d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24b8f1c4cdf54c36afbe48134cd210d8'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:44.826851 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942bcd7-c6b1-4564-a62d-1e29ceccdc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:44.827159 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942bcd7-c6b1-4564-a62d-1e29ceccdc8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:44.827835 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e942bcd7-c6b1-4564-a62d-1e29ceccdc8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:44.831088 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e942bcd7-c6b1-4564-a62d-1e29ceccdc8b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:44.841824 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-afbed27e-0063-40e6-81cb-a21ae31c15d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24b8f1c4cdf54c36afbe48134cd210d8'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:44.842415 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-afbed27e-0063-40e6-81cb-a21ae31c15d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24b8f1c4cdf54c36afbe48134cd210d8'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:44.849435 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1113/2221] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:29:44 2022] DELETE /identity/v3/users/24b8f1c4cdf54c36afbe48134cd210d8 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:29:44.854072 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1109/2222] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:29:44.854186 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbed27e-0063-40e6-81cb-a21ae31c15d6 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:44.856364 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aecf3e-28c0-40f6-adfa-df5a5e1b92dd None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:44.856523 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-afbed27e-0063-40e6-81cb-a21ae31c15d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:44.856620 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aecf3e-28c0-40f6-adfa-df5a5e1b92dd None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:44.856620 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aecf3e-28c0-40f6-adfa-df5a5e1b92dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:44.876053 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-00aecf3e-28c0-40f6-adfa-df5a5e1b92dd None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:44.877270 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-00aecf3e-28c0-40f6-adfa-df5a5e1b92dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:44.913968 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:44.930946 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbed27e-0063-40e6-81cb-a21ae31c15d6 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:44.936527 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fcde1a-2fd7-414b-9ea3-3e0fb83590af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:44.937581 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcde1a-2fd7-414b-9ea3-3e0fb83590af admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:44.938125 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcde1a-2fd7-414b-9ea3-3e0fb83590af admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:44.938794 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcde1a-2fd7-414b-9ea3-3e0fb83590af admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:44.939734 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fcde1a-2fd7-414b-9ea3-3e0fb83590af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:44.941609 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fcde1a-2fd7-414b-9ea3-3e0fb83590af admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:44.943621 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1110/2223] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:44.953917 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1114/2224] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:29:44.956618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aecf3e-28c0-40f6-adfa-df5a5e1b92dd None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:44.958113 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-00aecf3e-28c0-40f6-adfa-df5a5e1b92dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:44.959500 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fcde1a-2fd7-414b-9ea3-3e0fb83590af admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:44.969563 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fcde1a-2fd7-414b-9ea3-3e0fb83590af admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:44.974883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6a7aa3-3a92-46d9-b9ea-e9175d24d42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:44.975535 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6a7aa3-3a92-46d9-b9ea-e9175d24d42d admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:44.975821 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6a7aa3-3a92-46d9-b9ea-e9175d24d42d admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:44.976088 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6a7aa3-3a92-46d9-b9ea-e9175d24d42d admin admin] PATH_INFO: `/v3/projects/060fb9b1ecef4d48915cd004803b46e3/users/54867e83d2d04cbfbfec13d125b10fb3/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:44.991813 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6a7aa3-3a92-46d9-b9ea-e9175d24d42d admin admin] RBAC: Authorizing `identity:create_grant(project_id=060fb9b1ecef4d48915cd004803b46e3, user_id=54867e83d2d04cbfbfec13d125b10fb3, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:44.994519 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6a7aa3-3a92-46d9-b9ea-e9175d24d42d admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:45.020965 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1115/2225] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:29:44 2022] PUT /identity/v3/projects/060fb9b1ecef4d48915cd004803b46e3/users/54867e83d2d04cbfbfec13d125b10fb3/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:29:45.023102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc83ed-2548-4863-a71b-38ad21df50f3 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:45.023389 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc83ed-2548-4863-a71b-38ad21df50f3 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:45.023659 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc83ed-2548-4863-a71b-38ad21df50f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:45.033860 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aecf3e-28c0-40f6-adfa-df5a5e1b92dd None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:45.056573 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-73bc83ed-2548-4863-a71b-38ad21df50f3 None None] MFA Rules not processed for user `54867e83d2d04cbfbfec13d125b10fb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:45.058416 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-73bc83ed-2548-4863-a71b-38ad21df50f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:45.083314 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9cbfdd-381e-40ef-abf8-01cbaaf91438 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:45.083924 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9cbfdd-381e-40ef-abf8-01cbaaf91438 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:45.084236 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9cbfdd-381e-40ef-abf8-01cbaaf91438 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:45.084603 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9cbfdd-381e-40ef-abf8-01cbaaf91438 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:45.131622 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1116/2226] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:29:45.135973 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:45.137759 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc83ed-2548-4863-a71b-38ad21df50f3 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:45.157230 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9cbfdd-381e-40ef-abf8-01cbaaf91438 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:45.158735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9cbfdd-381e-40ef-abf8-01cbaaf91438 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:45.169979 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1111/2227] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:29:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 20:29:45.214821 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc83ed-2548-4863-a71b-38ad21df50f3 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:45.220336 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6c749-1345-466d-8706-7fd98607236a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:45.221099 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6c749-1345-466d-8706-7fd98607236a admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:45.221535 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6c749-1345-466d-8706-7fd98607236a admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:45.221960 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6c749-1345-466d-8706-7fd98607236a admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:45.222779 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc6c749-1345-466d-8706-7fd98607236a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:45.224749 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc6c749-1345-466d-8706-7fd98607236a admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:45.267204 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1117/2228] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:29:45 2022] POST /identity/v3/projects => generated 383 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:29:45.276104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9cbfdd-381e-40ef-abf8-01cbaaf91438 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:45.296770 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9cbfdd-381e-40ef-abf8-01cbaaf91438 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:45.416046 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5265b6-1910-46a9-b4c5-e758e8bc41dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:45.416675 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5265b6-1910-46a9-b4c5-e758e8bc41dd admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:45.417520 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5265b6-1910-46a9-b4c5-e758e8bc41dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:45.417933 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5265b6-1910-46a9-b4c5-e758e8bc41dd admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:45.418579 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5265b6-1910-46a9-b4c5-e758e8bc41dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:45.420797 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5265b6-1910-46a9-b4c5-e758e8bc41dd admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:45.453415 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1112/2229] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:29:45 2022] POST /identity/v3/users => generated 298 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:29:45.456278 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6c749-1345-466d-8706-7fd98607236a admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:45.472891 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6c749-1345-466d-8706-7fd98607236a admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:45.482776 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04ec4f-d3e7-4420-9deb-289916287bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:45.483827 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04ec4f-d3e7-4420-9deb-289916287bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:45.484408 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04ec4f-d3e7-4420-9deb-289916287bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:45.484863 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04ec4f-d3e7-4420-9deb-289916287bd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:45.485748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f04ec4f-d3e7-4420-9deb-289916287bd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:45.487618 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f04ec4f-d3e7-4420-9deb-289916287bd0 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:45.502861 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1118/2230] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:29:45.507155 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5265b6-1910-46a9-b4c5-e758e8bc41dd admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:45.525908 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04ec4f-d3e7-4420-9deb-289916287bd0 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:45.527646 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5265b6-1910-46a9-b4c5-e758e8bc41dd admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:45.528454 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-0f04ec4f-d3e7-4420-9deb-289916287bd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:45.541319 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce92e95-8fcb-4801-9354-a879dfe42d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:45.542551 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce92e95-8fcb-4801-9354-a879dfe42d96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:45.542888 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce92e95-8fcb-4801-9354-a879dfe42d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:45.543087 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce92e95-8fcb-4801-9354-a879dfe42d96 admin admin] PATH_INFO: `/v3/projects/a63bc70ae2fa4504a749058ad94e3b0d/users/6c2c5f09d4e1427bada2f2fc8790d50a/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:45.559548 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04ec4f-d3e7-4420-9deb-289916287bd0 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:45.572856 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d3e6a-8d82-4220-896e-3a75378648b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:45.573639 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d3e6a-8d82-4220-896e-3a75378648b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:45.574033 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d3e6a-8d82-4220-896e-3a75378648b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:45.574301 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d3e6a-8d82-4220-896e-3a75378648b5 admin admin] PATH_INFO: `/v3/projects/61f9c257416f456db4f2e0cbdfdfa616` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:45.575842 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243d3e6a-8d82-4220-896e-3a75378648b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61f9c257416f456db4f2e0cbdfdfa616)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:45.578658 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243d3e6a-8d82-4220-896e-3a75378648b5 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:45.588587 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce92e95-8fcb-4801-9354-a879dfe42d96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a63bc70ae2fa4504a749058ad94e3b0d, user_id=6c2c5f09d4e1427bada2f2fc8790d50a, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:45.593357 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce92e95-8fcb-4801-9354-a879dfe42d96 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:45.610819 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1113/2231] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:29:45 2022] PUT /identity/v3/projects/a63bc70ae2fa4504a749058ad94e3b0d/users/6c2c5f09d4e1427bada2f2fc8790d50a/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:45.617740 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce92e95-8fcb-4801-9354-a879dfe42d96 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:45.630925 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:45.690478 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-243d3e6a-8d82-4220-896e-3a75378648b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61f9c257416f456db4f2e0cbdfdfa616', 'request_id': 'req-243d3e6a-8d82-4220-896e-3a75378648b5'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:45.710036 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-243d3e6a-8d82-4220-896e-3a75378648b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61f9c257416f456db4f2e0cbdfdfa616', 'request_id': 'req-243d3e6a-8d82-4220-896e-3a75378648b5'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:45.715283 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce92e95-8fcb-4801-9354-a879dfe42d96 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:45.718719 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-243d3e6a-8d82-4220-896e-3a75378648b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61f9c257416f456db4f2e0cbdfdfa616', 'request_id': 'req-243d3e6a-8d82-4220-896e-3a75378648b5'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:45.744533 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-243d3e6a-8d82-4220-896e-3a75378648b5 admin admin] The token cache is being invalidate because project 61f9c257416f456db4f2e0cbdfdfa616 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:29:45.745510 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-243d3e6a-8d82-4220-896e-3a75378648b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:45.746268 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-243d3e6a-8d82-4220-896e-3a75378648b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:45.748425 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1119/2232] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:29:45 2022] DELETE /identity/v3/projects/61f9c257416f456db4f2e0cbdfdfa616 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:29:45.752732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d3e6a-8d82-4220-896e-3a75378648b5 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:45.754104 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-243d3e6a-8d82-4220-896e-3a75378648b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:45.767129 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:45.778568 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f629e-e6fc-4485-9e88-b31e69324a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:45.779183 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f629e-e6fc-4485-9e88-b31e69324a26 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:45.779459 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f629e-e6fc-4485-9e88-b31e69324a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:45.779727 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f629e-e6fc-4485-9e88-b31e69324a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:45.780369 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f629e-e6fc-4485-9e88-b31e69324a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:45.782132 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f629e-e6fc-4485-9e88-b31e69324a26 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:45.792813 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1114/2233] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:29:45.797427 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f629e-e6fc-4485-9e88-b31e69324a26 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:45.799136 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-570f629e-e6fc-4485-9e88-b31e69324a26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:45.812460 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f629e-e6fc-4485-9e88-b31e69324a26 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:45.817616 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc60a0-d6b6-4096-9917-dca7b194cebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:45.818285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc60a0-d6b6-4096-9917-dca7b194cebc admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:45.818695 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc60a0-d6b6-4096-9917-dca7b194cebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:45.818983 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc60a0-d6b6-4096-9917-dca7b194cebc admin admin] PATH_INFO: `/v3/projects/a63bc70ae2fa4504a749058ad94e3b0d/users/6c2c5f09d4e1427bada2f2fc8790d50a/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:45.821192 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbc60a0-d6b6-4096-9917-dca7b194cebc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a63bc70ae2fa4504a749058ad94e3b0d, user_id=6c2c5f09d4e1427bada2f2fc8790d50a, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:45.824119 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbc60a0-d6b6-4096-9917-dca7b194cebc admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:45.827286 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d3e6a-8d82-4220-896e-3a75378648b5 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:45.832525 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f74f17d-8d53-44cf-83e9-57afd74a1e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:45.833264 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74f17d-8d53-44cf-83e9-57afd74a1e32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:45.833619 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74f17d-8d53-44cf-83e9-57afd74a1e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:45.833965 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74f17d-8d53-44cf-83e9-57afd74a1e32 admin admin] PATH_INFO: `/v3/users/1beac350ec7043b287acbfde6996d942` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:45.835175 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f74f17d-8d53-44cf-83e9-57afd74a1e32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1beac350ec7043b287acbfde6996d942)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:45.837287 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f74f17d-8d53-44cf-83e9-57afd74a1e32 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:45.841319 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1115/2234] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:29:45 2022] PUT /identity/v3/projects/a63bc70ae2fa4504a749058ad94e3b0d/users/6c2c5f09d4e1427bada2f2fc8790d50a/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:29:45.847584 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb72aa6-d78a-4046-a1b9-007f04606ec7 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:45.847867 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb72aa6-d78a-4046-a1b9-007f04606ec7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:45.848140 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb72aa6-d78a-4046-a1b9-007f04606ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:45.877419 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0fb72aa6-d78a-4046-a1b9-007f04606ec7 None None] MFA Rules not processed for user `6c2c5f09d4e1427bada2f2fc8790d50a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:45.878785 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0fb72aa6-d78a-4046-a1b9-007f04606ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:45.896054 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-3f74f17d-8d53-44cf-83e9-57afd74a1e32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1beac350ec7043b287acbfde6996d942'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:45.910781 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-3f74f17d-8d53-44cf-83e9-57afd74a1e32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1beac350ec7043b287acbfde6996d942'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:45.918687 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-3f74f17d-8d53-44cf-83e9-57afd74a1e32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1beac350ec7043b287acbfde6996d942'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:45.927458 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-3f74f17d-8d53-44cf-83e9-57afd74a1e32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1beac350ec7043b287acbfde6996d942'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:45.928888 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-3f74f17d-8d53-44cf-83e9-57afd74a1e32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1beac350ec7043b287acbfde6996d942'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:45.936218 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1120/2235] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:29:45 2022] DELETE /identity/v3/users/1beac350ec7043b287acbfde6996d942 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:45.954302 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1116/2236] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:45 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:45.966243 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f74f17d-8d53-44cf-83e9-57afd74a1e32 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:45.968037 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-3f74f17d-8d53-44cf-83e9-57afd74a1e32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:46.015341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:46.052602 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f74f17d-8d53-44cf-83e9-57afd74a1e32 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:46.057313 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec7922-d055-4146-b7ee-f197782944cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:46.058178 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec7922-d055-4146-b7ee-f197782944cf service cinder] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:46.058523 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec7922-d055-4146-b7ee-f197782944cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:46.058800 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec7922-d055-4146-b7ee-f197782944cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:46.060660 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-1dec7922-d055-4146-b7ee-f197782944cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:46.145843 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dec7922-d055-4146-b7ee-f197782944cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:46.147289 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dec7922-d055-4146-b7ee-f197782944cf service cinder] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:46.158476 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1121/2237] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:29:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:29:46.252597 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb72aa6-d78a-4046-a1b9-007f04606ec7 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:46.254228 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0fb72aa6-d78a-4046-a1b9-007f04606ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:46.399969 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb72aa6-d78a-4046-a1b9-007f04606ec7 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:46.469494 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf6ee7-8c5d-4233-8559-cf123d466080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:46.470537 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf6ee7-8c5d-4233-8559-cf123d466080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:46.470537 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf6ee7-8c5d-4233-8559-cf123d466080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:46.470695 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf6ee7-8c5d-4233-8559-cf123d466080 admin admin] PATH_INFO: `/v3/projects/20a2cfd8cde44efd9147d9a792de7341` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:46.471851 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecf6ee7-8c5d-4233-8559-cf123d466080 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20a2cfd8cde44efd9147d9a792de7341)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:46.473989 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecf6ee7-8c5d-4233-8559-cf123d466080 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:46.487190 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:46.587002 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3ecf6ee7-8c5d-4233-8559-cf123d466080 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20a2cfd8cde44efd9147d9a792de7341', 'request_id': 'req-3ecf6ee7-8c5d-4233-8559-cf123d466080'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:46.604552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3ecf6ee7-8c5d-4233-8559-cf123d466080 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20a2cfd8cde44efd9147d9a792de7341', 'request_id': 'req-3ecf6ee7-8c5d-4233-8559-cf123d466080'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:46.613829 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3ecf6ee7-8c5d-4233-8559-cf123d466080 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20a2cfd8cde44efd9147d9a792de7341', 'request_id': 'req-3ecf6ee7-8c5d-4233-8559-cf123d466080'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:46.633455 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3ecf6ee7-8c5d-4233-8559-cf123d466080 admin admin] The token cache is being invalidate because project 20a2cfd8cde44efd9147d9a792de7341 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:29:46.633893 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3ecf6ee7-8c5d-4233-8559-cf123d466080 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:46.634607 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3ecf6ee7-8c5d-4233-8559-cf123d466080 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:46.641338 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1117/2238] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:29:46 2022] DELETE /identity/v3/projects/20a2cfd8cde44efd9147d9a792de7341 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:46.642732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c84aed4-2382-4466-a21d-c7057fe277f9 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:46.643581 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c84aed4-2382-4466-a21d-c7057fe277f9 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:46.643934 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c84aed4-2382-4466-a21d-c7057fe277f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:46.674426 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-4c84aed4-2382-4466-a21d-c7057fe277f9 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:46.675853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-4c84aed4-2382-4466-a21d-c7057fe277f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:46.751025 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1122/2239] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:29:46.759442 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade65665-7d56-4baf-a4bc-e7f97e58c3aa None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:46.759802 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade65665-7d56-4baf-a4bc-e7f97e58c3aa None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:46.760146 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade65665-7d56-4baf-a4bc-e7f97e58c3aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:46.783086 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ade65665-7d56-4baf-a4bc-e7f97e58c3aa None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:46.784515 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ade65665-7d56-4baf-a4bc-e7f97e58c3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:46.799873 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c84aed4-2382-4466-a21d-c7057fe277f9 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:46.806379 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1118/2240] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:46.812939 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c84aed4-2382-4466-a21d-c7057fe277f9 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:46.824242 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f5d43-92ba-445d-be05-5f980b82d96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:46.824977 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f5d43-92ba-445d-be05-5f980b82d96e admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:46.825356 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f5d43-92ba-445d-be05-5f980b82d96e admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:46.826651 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f5d43-92ba-445d-be05-5f980b82d96e admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:46.827401 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522f5d43-92ba-445d-be05-5f980b82d96e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:29:46.828753 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522f5d43-92ba-445d-be05-5f980b82d96e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:46.831583 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522f5d43-92ba-445d-be05-5f980b82d96e admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:46.847408 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1123/2241] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:29:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:29:46.852070 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade65665-7d56-4baf-a4bc-e7f97e58c3aa None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:46.865217 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade65665-7d56-4baf-a4bc-e7f97e58c3aa None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:46.871554 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf201ce-28b3-4906-a4e2-5c8cf741935f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:46.872190 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf201ce-28b3-4906-a4e2-5c8cf741935f admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:46.872471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf201ce-28b3-4906-a4e2-5c8cf741935f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:46.872743 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf201ce-28b3-4906-a4e2-5c8cf741935f admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:46.873407 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf201ce-28b3-4906-a4e2-5c8cf741935f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:46.875585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf201ce-28b3-4906-a4e2-5c8cf741935f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:46.912902 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1119/2242] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:29:46 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:29:46.918040 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f5d43-92ba-445d-be05-5f980b82d96e admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:46.929438 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f5d43-92ba-445d-be05-5f980b82d96e admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:47.015111 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b7f1f-f44f-499e-aff2-33d28c744587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:47.015821 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b7f1f-f44f-499e-aff2-33d28c744587 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:47.016102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b7f1f-f44f-499e-aff2-33d28c744587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:47.016373 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b7f1f-f44f-499e-aff2-33d28c744587 admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:47.017003 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6b7f1f-f44f-499e-aff2-33d28c744587 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:47.019037 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6b7f1f-f44f-499e-aff2-33d28c744587 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:47.043532 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1124/2243] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:29:46 2022] POST /identity/v3/users => generated 288 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:29:47.044729 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf201ce-28b3-4906-a4e2-5c8cf741935f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:47.055428 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf201ce-28b3-4906-a4e2-5c8cf741935f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:47.060139 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8219981c-dd2e-4e85-8f63-cbad0cd0c62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:47.060986 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8219981c-dd2e-4e85-8f63-cbad0cd0c62e admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:47.061298 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8219981c-dd2e-4e85-8f63-cbad0cd0c62e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:47.061573 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8219981c-dd2e-4e85-8f63-cbad0cd0c62e admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:47.062444 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8219981c-dd2e-4e85-8f63-cbad0cd0c62e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:47.064273 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8219981c-dd2e-4e85-8f63-cbad0cd0c62e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:47.076762 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1120/2244] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:29:47.080787 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b7f1f-f44f-499e-aff2-33d28c744587 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:47.086019 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:47.092895 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b7f1f-f44f-499e-aff2-33d28c744587 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:47.098371 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5341e0bb-d5bf-4377-b2bf-360b6e8967d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:47.098974 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341e0bb-d5bf-4377-b2bf-360b6e8967d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:47.099253 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341e0bb-d5bf-4377-b2bf-360b6e8967d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:47.099523 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341e0bb-d5bf-4377-b2bf-360b6e8967d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:47.100169 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5341e0bb-d5bf-4377-b2bf-360b6e8967d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:47.101798 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5341e0bb-d5bf-4377-b2bf-360b6e8967d9 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:47.114216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1125/2245] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:29:47.118657 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8219981c-dd2e-4e85-8f63-cbad0cd0c62e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:47.129221 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8219981c-dd2e-4e85-8f63-cbad0cd0c62e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:47.133925 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8409fc-3e65-4ac7-9426-87352a23ba3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:47.134577 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8409fc-3e65-4ac7-9426-87352a23ba3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:47.134857 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8409fc-3e65-4ac7-9426-87352a23ba3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:47.135264 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8409fc-3e65-4ac7-9426-87352a23ba3b admin admin] PATH_INFO: `/v3/projects/51f7d6271f574a97a9277e584092b425/users/929aaf82cf4b42caaa615f6bed53519d/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:47.150073 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8409fc-3e65-4ac7-9426-87352a23ba3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=51f7d6271f574a97a9277e584092b425, user_id=929aaf82cf4b42caaa615f6bed53519d, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:47.152823 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8409fc-3e65-4ac7-9426-87352a23ba3b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:47.165824 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1121/2246] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:29:47 2022] PUT /identity/v3/projects/51f7d6271f574a97a9277e584092b425/users/929aaf82cf4b42caaa615f6bed53519d/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:47.178036 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27119284-1951-4425-beec-86d8487069e8 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:47.178321 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27119284-1951-4425-beec-86d8487069e8 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:47.178618 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27119284-1951-4425-beec-86d8487069e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:47.211304 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-27119284-1951-4425-beec-86d8487069e8 None None] MFA Rules not processed for user `929aaf82cf4b42caaa615f6bed53519d`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:47.212521 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-27119284-1951-4425-beec-86d8487069e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:47.274814 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1126/2247] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:47.286283 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8409fc-3e65-4ac7-9426-87352a23ba3b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:47.287978 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1e8409fc-3e65-4ac7-9426-87352a23ba3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:47.344624 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8409fc-3e65-4ac7-9426-87352a23ba3b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:47.348243 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78a550-88f1-42da-ad80-34b592697f52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:47.348845 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78a550-88f1-42da-ad80-34b592697f52 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:47.349123 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78a550-88f1-42da-ad80-34b592697f52 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:47.349437 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78a550-88f1-42da-ad80-34b592697f52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:47.357836 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d78a550-88f1-42da-ad80-34b592697f52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:47.359252 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d78a550-88f1-42da-ad80-34b592697f52 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:47.369073 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1122/2248] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:29:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:29:47.841431 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27119284-1951-4425-beec-86d8487069e8 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:47.842921 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-27119284-1951-4425-beec-86d8487069e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:47.855057 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27119284-1951-4425-beec-86d8487069e8 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:47.858572 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81099d03-00da-4f44-807c-4de1d01d4921 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:47.859164 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81099d03-00da-4f44-807c-4de1d01d4921 service cinder] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:47.859439 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81099d03-00da-4f44-807c-4de1d01d4921 service cinder] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:47.859705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81099d03-00da-4f44-807c-4de1d01d4921 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:47.861376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-81099d03-00da-4f44-807c-4de1d01d4921 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:47.924488 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81099d03-00da-4f44-807c-4de1d01d4921 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:47.926136 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81099d03-00da-4f44-807c-4de1d01d4921 service cinder] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:47.935910 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1127/2249] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:29:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 20:29:48.566516 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78a550-88f1-42da-ad80-34b592697f52 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:48.575317 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78a550-88f1-42da-ad80-34b592697f52 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:48.578975 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea22792-9f48-44a4-b30d-1d12b46a679c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:48.579569 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea22792-9f48-44a4-b30d-1d12b46a679c service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:48.579887 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea22792-9f48-44a4-b30d-1d12b46a679c service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:48.580164 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea22792-9f48-44a4-b30d-1d12b46a679c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:48.581930 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0ea22792-9f48-44a4-b30d-1d12b46a679c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:48.618544 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:48.652318 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea22792-9f48-44a4-b30d-1d12b46a679c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:48.654150 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea22792-9f48-44a4-b30d-1d12b46a679c service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:48.665392 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1123/2250] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:29:48 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:29:55.842429 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81099d03-00da-4f44-807c-4de1d01d4921 service cinder] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:55.843899 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-81099d03-00da-4f44-807c-4de1d01d4921 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:55.907663 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81099d03-00da-4f44-807c-4de1d01d4921 service cinder] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:55.912142 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7d77f-3c5b-48d6-9afe-72c3ad2e5b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:55.912767 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7d77f-3c5b-48d6-9afe-72c3ad2e5b1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:55.913044 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7d77f-3c5b-48d6-9afe-72c3ad2e5b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:55.913347 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7d77f-3c5b-48d6-9afe-72c3ad2e5b1b admin admin] PATH_INFO: `/v3/users/1a37cb0446c842ebaedebdf1ec08e265` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:55.914445 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f7d77f-3c5b-48d6-9afe-72c3ad2e5b1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a37cb0446c842ebaedebdf1ec08e265)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:55.916859 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f7d77f-3c5b-48d6-9afe-72c3ad2e5b1b admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:55.937704 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:55.961605 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-f7f7d77f-3c5b-48d6-9afe-72c3ad2e5b1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a37cb0446c842ebaedebdf1ec08e265'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:55.973361 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-f7f7d77f-3c5b-48d6-9afe-72c3ad2e5b1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a37cb0446c842ebaedebdf1ec08e265'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:55.980673 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-f7f7d77f-3c5b-48d6-9afe-72c3ad2e5b1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a37cb0446c842ebaedebdf1ec08e265'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:55.987864 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-f7f7d77f-3c5b-48d6-9afe-72c3ad2e5b1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a37cb0446c842ebaedebdf1ec08e265'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:55.988559 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-f7f7d77f-3c5b-48d6-9afe-72c3ad2e5b1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a37cb0446c842ebaedebdf1ec08e265'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:55.995302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1128/2251] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:29:55 2022] DELETE /identity/v3/users/1a37cb0446c842ebaedebdf1ec08e265 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:55.999807 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c38c4-b14a-4103-a8f5-58d262e1df34 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:56.000326 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c38c4-b14a-4103-a8f5-58d262e1df34 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:56.000431 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c38c4-b14a-4103-a8f5-58d262e1df34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:56.019012 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-2e4c38c4-b14a-4103-a8f5-58d262e1df34 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:56.020277 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2e4c38c4-b14a-4103-a8f5-58d262e1df34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:56.079121 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1124/2252] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:29:56.089308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7d77f-3c5b-48d6-9afe-72c3ad2e5b1b admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:56.090813 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-f7f7d77f-3c5b-48d6-9afe-72c3ad2e5b1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:56.152246 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7d77f-3c5b-48d6-9afe-72c3ad2e5b1b admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:56.155924 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ddb954-598e-4303-899e-6ade325092b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:56.156685 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddb954-598e-4303-899e-6ade325092b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:56.156970 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddb954-598e-4303-899e-6ade325092b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:56.157262 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddb954-598e-4303-899e-6ade325092b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:56.166394 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ddb954-598e-4303-899e-6ade325092b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:56.167826 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ddb954-598e-4303-899e-6ade325092b7 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:56.178313 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1129/2253] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:29:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:29:56.506921 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c38c4-b14a-4103-a8f5-58d262e1df34 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:56.508401 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2e4c38c4-b14a-4103-a8f5-58d262e1df34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:56.519868 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ddb954-598e-4303-899e-6ade325092b7 service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:56.521768 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-81ddb954-598e-4303-899e-6ade325092b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:56.525687 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c38c4-b14a-4103-a8f5-58d262e1df34 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:56.535898 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3b922-5a9b-4f47-8a15-fadaf30ef3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:56.536679 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3b922-5a9b-4f47-8a15-fadaf30ef3ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:56.537033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3b922-5a9b-4f47-8a15-fadaf30ef3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:56.537405 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3b922-5a9b-4f47-8a15-fadaf30ef3ab admin admin] PATH_INFO: `/v3/projects/65c0a823d2db4790be0da2553b43a92c` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:56.538635 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c3b922-5a9b-4f47-8a15-fadaf30ef3ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=65c0a823d2db4790be0da2553b43a92c)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:56.539522 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ddb954-598e-4303-899e-6ade325092b7 service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:56.540982 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c3b922-5a9b-4f47-8a15-fadaf30ef3ab admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:56.546745 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e4614a-1e35-4d99-90ae-3f5fb9a97015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:56.546929 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e4614a-1e35-4d99-90ae-3f5fb9a97015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:56.547162 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e4614a-1e35-4d99-90ae-3f5fb9a97015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:56.547438 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e4614a-1e35-4d99-90ae-3f5fb9a97015 admin admin] PATH_INFO: `/v3/users/dac1bb313ddf435181219337588777cc` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:56.548722 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e4614a-1e35-4d99-90ae-3f5fb9a97015 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dac1bb313ddf435181219337588777cc)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:56.551662 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e4614a-1e35-4d99-90ae-3f5fb9a97015 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:56.636297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c5c3b922-5a9b-4f47-8a15-fadaf30ef3ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65c0a823d2db4790be0da2553b43a92c', 'request_id': 'req-c5c3b922-5a9b-4f47-8a15-fadaf30ef3ab'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:56.651857 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c5c3b922-5a9b-4f47-8a15-fadaf30ef3ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65c0a823d2db4790be0da2553b43a92c', 'request_id': 'req-c5c3b922-5a9b-4f47-8a15-fadaf30ef3ab'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:56.660699 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c5c3b922-5a9b-4f47-8a15-fadaf30ef3ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65c0a823d2db4790be0da2553b43a92c', 'request_id': 'req-c5c3b922-5a9b-4f47-8a15-fadaf30ef3ab'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:56.665501 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-72e4614a-1e35-4d99-90ae-3f5fb9a97015 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dac1bb313ddf435181219337588777cc'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:56.680705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-72e4614a-1e35-4d99-90ae-3f5fb9a97015 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dac1bb313ddf435181219337588777cc'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:56.686775 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c5c3b922-5a9b-4f47-8a15-fadaf30ef3ab admin admin] The token cache is being invalidate because project 65c0a823d2db4790be0da2553b43a92c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:29:56.687210 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c5c3b922-5a9b-4f47-8a15-fadaf30ef3ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:56.687932 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c5c3b922-5a9b-4f47-8a15-fadaf30ef3ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:56.688545 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-72e4614a-1e35-4d99-90ae-3f5fb9a97015 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dac1bb313ddf435181219337588777cc'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:56.689668 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1125/2254] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:29:56 2022] DELETE /identity/v3/projects/65c0a823d2db4790be0da2553b43a92c => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:56.693769 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3b922-5a9b-4f47-8a15-fadaf30ef3ab admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:56.695441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c5c3b922-5a9b-4f47-8a15-fadaf30ef3ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:56.697734 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-72e4614a-1e35-4d99-90ae-3f5fb9a97015 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dac1bb313ddf435181219337588777cc'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:56.698696 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-72e4614a-1e35-4d99-90ae-3f5fb9a97015 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dac1bb313ddf435181219337588777cc'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:56.712866 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1130/2255] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:29:56 2022] DELETE /identity/v3/users/dac1bb313ddf435181219337588777cc => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:29:56.717341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925428f-f569-40d6-a0cf-26c6e49fee33 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:56.717633 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925428f-f569-40d6-a0cf-26c6e49fee33 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:56.717907 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925428f-f569-40d6-a0cf-26c6e49fee33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:56.739123 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-5925428f-f569-40d6-a0cf-26c6e49fee33 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:56.740594 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-5925428f-f569-40d6-a0cf-26c6e49fee33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:56.749113 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:56.765915 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:56.770502 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3b922-5a9b-4f47-8a15-fadaf30ef3ab admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:56.805966 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1131/2256] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:56.816380 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925428f-f569-40d6-a0cf-26c6e49fee33 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:56.817841 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-5925428f-f569-40d6-a0cf-26c6e49fee33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:56.841611 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a4b1f4-105b-4bfd-ace8-a9209f55868d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:56.842249 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a4b1f4-105b-4bfd-ace8-a9209f55868d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:56.842584 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a4b1f4-105b-4bfd-ace8-a9209f55868d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:56.842893 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a4b1f4-105b-4bfd-ace8-a9209f55868d admin admin] PATH_INFO: `/v3/users/54867e83d2d04cbfbfec13d125b10fb3` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:56.844067 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a4b1f4-105b-4bfd-ace8-a9209f55868d admin admin] RBAC: Authorizing `identity:delete_user(user_id=54867e83d2d04cbfbfec13d125b10fb3)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:56.846717 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a4b1f4-105b-4bfd-ace8-a9209f55868d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:56.879365 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925428f-f569-40d6-a0cf-26c6e49fee33 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:56.883475 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8b572-ddfd-41c6-bc71-5cca3baa7f59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:56.884092 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8b572-ddfd-41c6-bc71-5cca3baa7f59 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:56.884478 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8b572-ddfd-41c6-bc71-5cca3baa7f59 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:56.884754 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8b572-ddfd-41c6-bc71-5cca3baa7f59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:56.894580 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa8b572-ddfd-41c6-bc71-5cca3baa7f59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:56.896016 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa8b572-ddfd-41c6-bc71-5cca3baa7f59 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:56.896966 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-26a4b1f4-105b-4bfd-ace8-a9209f55868d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54867e83d2d04cbfbfec13d125b10fb3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:56.905378 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-26a4b1f4-105b-4bfd-ace8-a9209f55868d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54867e83d2d04cbfbfec13d125b10fb3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:56.906533 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1132/2257] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:29:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 20:29:56.922661 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-26a4b1f4-105b-4bfd-ace8-a9209f55868d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54867e83d2d04cbfbfec13d125b10fb3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:56.931718 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-26a4b1f4-105b-4bfd-ace8-a9209f55868d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54867e83d2d04cbfbfec13d125b10fb3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:56.933047 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-26a4b1f4-105b-4bfd-ace8-a9209f55868d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54867e83d2d04cbfbfec13d125b10fb3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:56.940078 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1126/2258] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:29:56 2022] DELETE /identity/v3/users/54867e83d2d04cbfbfec13d125b10fb3 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:29:57.255538 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8b572-ddfd-41c6-bc71-5cca3baa7f59 service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:57.257542 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-caa8b572-ddfd-41c6-bc71-5cca3baa7f59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:57.272450 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8b572-ddfd-41c6-bc71-5cca3baa7f59 service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:57.326339 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a4b1f4-105b-4bfd-ace8-a9209f55868d admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:57.328438 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-26a4b1f4-105b-4bfd-ace8-a9209f55868d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:57.372882 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6696469e-9a71-42ba-8b33-b55a38a978ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:57.374188 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696469e-9a71-42ba-8b33-b55a38a978ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:57.374188 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696469e-9a71-42ba-8b33-b55a38a978ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:57.374536 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696469e-9a71-42ba-8b33-b55a38a978ac admin admin] PATH_INFO: `/v3/projects/9b9d6b789ba942ef8afd5821269a2782` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:57.376430 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6696469e-9a71-42ba-8b33-b55a38a978ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b9d6b789ba942ef8afd5821269a2782)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:57.378339 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6696469e-9a71-42ba-8b33-b55a38a978ac admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:57.419597 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a4b1f4-105b-4bfd-ace8-a9209f55868d admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:57.434446 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6696469e-9a71-42ba-8b33-b55a38a978ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b9d6b789ba942ef8afd5821269a2782', 'request_id': 'req-6696469e-9a71-42ba-8b33-b55a38a978ac'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:57.442452 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6696469e-9a71-42ba-8b33-b55a38a978ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b9d6b789ba942ef8afd5821269a2782', 'request_id': 'req-6696469e-9a71-42ba-8b33-b55a38a978ac'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:57.444803 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:57.450488 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:57.455143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6696469e-9a71-42ba-8b33-b55a38a978ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b9d6b789ba942ef8afd5821269a2782', 'request_id': 'req-6696469e-9a71-42ba-8b33-b55a38a978ac'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:57.468390 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6696469e-9a71-42ba-8b33-b55a38a978ac admin admin] The token cache is being invalidate because project 9b9d6b789ba942ef8afd5821269a2782 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:29:57.468681 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6696469e-9a71-42ba-8b33-b55a38a978ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:57.469374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6696469e-9a71-42ba-8b33-b55a38a978ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:57.471573 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1133/2259] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:29:57 2022] DELETE /identity/v3/projects/9b9d6b789ba942ef8afd5821269a2782 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:57.475253 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6696469e-9a71-42ba-8b33-b55a38a978ac admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:57.476594 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6696469e-9a71-42ba-8b33-b55a38a978ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:57.484693 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82092d09-cd49-4f23-854b-69e129b8cbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:57.485604 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82092d09-cd49-4f23-854b-69e129b8cbd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:57.485958 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82092d09-cd49-4f23-854b-69e129b8cbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:57.486330 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82092d09-cd49-4f23-854b-69e129b8cbd3 admin admin] PATH_INFO: `/v3/projects/060fb9b1ecef4d48915cd004803b46e3` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:57.487988 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82092d09-cd49-4f23-854b-69e129b8cbd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=060fb9b1ecef4d48915cd004803b46e3)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:57.490354 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82092d09-cd49-4f23-854b-69e129b8cbd3 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:57.552384 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6696469e-9a71-42ba-8b33-b55a38a978ac admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:57.553543 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-82092d09-cd49-4f23-854b-69e129b8cbd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '060fb9b1ecef4d48915cd004803b46e3', 'request_id': 'req-82092d09-cd49-4f23-854b-69e129b8cbd3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:57.559873 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-82092d09-cd49-4f23-854b-69e129b8cbd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '060fb9b1ecef4d48915cd004803b46e3', 'request_id': 'req-82092d09-cd49-4f23-854b-69e129b8cbd3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:57.566480 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-82092d09-cd49-4f23-854b-69e129b8cbd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '060fb9b1ecef4d48915cd004803b46e3', 'request_id': 'req-82092d09-cd49-4f23-854b-69e129b8cbd3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:57.579078 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-82092d09-cd49-4f23-854b-69e129b8cbd3 admin admin] The token cache is being invalidate because project 060fb9b1ecef4d48915cd004803b46e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:29:57.579360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-82092d09-cd49-4f23-854b-69e129b8cbd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:57.579974 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-82092d09-cd49-4f23-854b-69e129b8cbd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:57.581579 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1127/2260] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:29:57 2022] DELETE /identity/v3/projects/060fb9b1ecef4d48915cd004803b46e3 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:57.585171 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82092d09-cd49-4f23-854b-69e129b8cbd3 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:57.586635 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-82092d09-cd49-4f23-854b-69e129b8cbd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:57.612367 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb7689c-a5e4-4f71-bd43-51f1d77fd006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:57.613010 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7689c-a5e4-4f71-bd43-51f1d77fd006 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:57.613315 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7689c-a5e4-4f71-bd43-51f1d77fd006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:57.613594 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7689c-a5e4-4f71-bd43-51f1d77fd006 admin admin] PATH_INFO: `/v3/users/fa3733fe2760466d945b82c2fad415ec` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:57.615255 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb7689c-a5e4-4f71-bd43-51f1d77fd006 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa3733fe2760466d945b82c2fad415ec)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:57.617189 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb7689c-a5e4-4f71-bd43-51f1d77fd006 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:57.653965 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82092d09-cd49-4f23-854b-69e129b8cbd3 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:57.662442 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b026f7ea-6b35-4e2e-a625-f7dce30bfa6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:57.663201 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026f7ea-6b35-4e2e-a625-f7dce30bfa6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:57.663585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026f7ea-6b35-4e2e-a625-f7dce30bfa6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:57.664144 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026f7ea-6b35-4e2e-a625-f7dce30bfa6d admin admin] PATH_INFO: `/v3/users/6c2c5f09d4e1427bada2f2fc8790d50a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:57.665437 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b026f7ea-6b35-4e2e-a625-f7dce30bfa6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c2c5f09d4e1427bada2f2fc8790d50a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:57.673348 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b026f7ea-6b35-4e2e-a625-f7dce30bfa6d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:57.682482 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-bdb7689c-a5e4-4f71-bd43-51f1d77fd006 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa3733fe2760466d945b82c2fad415ec'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:57.700070 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-bdb7689c-a5e4-4f71-bd43-51f1d77fd006 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa3733fe2760466d945b82c2fad415ec'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:57.714586 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-bdb7689c-a5e4-4f71-bd43-51f1d77fd006 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa3733fe2760466d945b82c2fad415ec'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:57.729536 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-bdb7689c-a5e4-4f71-bd43-51f1d77fd006 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa3733fe2760466d945b82c2fad415ec'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:57.730880 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-bdb7689c-a5e4-4f71-bd43-51f1d77fd006 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa3733fe2760466d945b82c2fad415ec'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:57.735134 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b026f7ea-6b35-4e2e-a625-f7dce30bfa6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c2c5f09d4e1427bada2f2fc8790d50a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:57.743001 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1134/2261] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:29:57 2022] DELETE /identity/v3/users/fa3733fe2760466d945b82c2fad415ec => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:29:57.743652 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b026f7ea-6b35-4e2e-a625-f7dce30bfa6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c2c5f09d4e1427bada2f2fc8790d50a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:57.764164 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:57.782996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b026f7ea-6b35-4e2e-a625-f7dce30bfa6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c2c5f09d4e1427bada2f2fc8790d50a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:57.795496 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b026f7ea-6b35-4e2e-a625-f7dce30bfa6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c2c5f09d4e1427bada2f2fc8790d50a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:57.797800 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b026f7ea-6b35-4e2e-a625-f7dce30bfa6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c2c5f09d4e1427bada2f2fc8790d50a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:57.809301 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1128/2262] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:29:57 2022] DELETE /identity/v3/users/6c2c5f09d4e1427bada2f2fc8790d50a => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:29:58.245927 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb7689c-a5e4-4f71-bd43-51f1d77fd006 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:58.247373 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-bdb7689c-a5e4-4f71-bd43-51f1d77fd006 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:58.273370 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:58.336679 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb7689c-a5e4-4f71-bd43-51f1d77fd006 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:58.417873 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ded2b4-230a-459a-900c-f8d80f8507cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:58.418514 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ded2b4-230a-459a-900c-f8d80f8507cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:58.418804 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ded2b4-230a-459a-900c-f8d80f8507cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:58.419070 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ded2b4-230a-459a-900c-f8d80f8507cc admin admin] PATH_INFO: `/v3/projects/6d518257591540da9f6d1ba2ea5df4d2` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:58.420203 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ded2b4-230a-459a-900c-f8d80f8507cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d518257591540da9f6d1ba2ea5df4d2)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:58.422210 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ded2b4-230a-459a-900c-f8d80f8507cc admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:58.487727 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-a2ded2b4-230a-459a-900c-f8d80f8507cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d518257591540da9f6d1ba2ea5df4d2', 'request_id': 'req-a2ded2b4-230a-459a-900c-f8d80f8507cc'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:58.496793 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-a2ded2b4-230a-459a-900c-f8d80f8507cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d518257591540da9f6d1ba2ea5df4d2', 'request_id': 'req-a2ded2b4-230a-459a-900c-f8d80f8507cc'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:58.503304 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-a2ded2b4-230a-459a-900c-f8d80f8507cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d518257591540da9f6d1ba2ea5df4d2', 'request_id': 'req-a2ded2b4-230a-459a-900c-f8d80f8507cc'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:58.518151 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b026f7ea-6b35-4e2e-a625-f7dce30bfa6d admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:58.520334 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b026f7ea-6b35-4e2e-a625-f7dce30bfa6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:58.521746 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-a2ded2b4-230a-459a-900c-f8d80f8507cc admin admin] The token cache is being invalidate because project 6d518257591540da9f6d1ba2ea5df4d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:29:58.522270 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-a2ded2b4-230a-459a-900c-f8d80f8507cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:58.524304 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-a2ded2b4-230a-459a-900c-f8d80f8507cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:58.527014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1135/2263] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:29:58 2022] DELETE /identity/v3/projects/6d518257591540da9f6d1ba2ea5df4d2 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:58.531864 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1040c71-3368-47ec-be4f-1fd915205084 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:58.532350 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1040c71-3368-47ec-be4f-1fd915205084 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:58.532764 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1040c71-3368-47ec-be4f-1fd915205084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:58.544360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b026f7ea-6b35-4e2e-a625-f7dce30bfa6d admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:58.589444 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-f1040c71-3368-47ec-be4f-1fd915205084 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:58.589444 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-f1040c71-3368-47ec-be4f-1fd915205084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:58.641936 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd3006f-9013-42cd-9bd6-335a07c91b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:58.642942 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3006f-9013-42cd-9bd6-335a07c91b04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:58.643056 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:58.643566 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3006f-9013-42cd-9bd6-335a07c91b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:58.644027 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3006f-9013-42cd-9bd6-335a07c91b04 admin admin] PATH_INFO: `/v3/projects/a63bc70ae2fa4504a749058ad94e3b0d` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:58.645492 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd3006f-9013-42cd-9bd6-335a07c91b04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a63bc70ae2fa4504a749058ad94e3b0d)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:58.648556 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd3006f-9013-42cd-9bd6-335a07c91b04 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:58.670255 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1136/2264] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:58.676143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557e855-1151-4c2d-9684-5bd6d5581af6 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:58.676143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557e855-1151-4c2d-9684-5bd6d5581af6 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:58.676143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557e855-1151-4c2d-9684-5bd6d5581af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:58.709862 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-4557e855-1151-4c2d-9684-5bd6d5581af6 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:58.711370 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-4557e855-1151-4c2d-9684-5bd6d5581af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:58.738352 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3fd3006f-9013-42cd-9bd6-335a07c91b04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a63bc70ae2fa4504a749058ad94e3b0d', 'request_id': 'req-3fd3006f-9013-42cd-9bd6-335a07c91b04'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:58.748207 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3fd3006f-9013-42cd-9bd6-335a07c91b04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a63bc70ae2fa4504a749058ad94e3b0d', 'request_id': 'req-3fd3006f-9013-42cd-9bd6-335a07c91b04'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:58.768078 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3fd3006f-9013-42cd-9bd6-335a07c91b04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a63bc70ae2fa4504a749058ad94e3b0d', 'request_id': 'req-3fd3006f-9013-42cd-9bd6-335a07c91b04'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:58.786739 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3fd3006f-9013-42cd-9bd6-335a07c91b04 admin admin] The token cache is being invalidate because project a63bc70ae2fa4504a749058ad94e3b0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:29:58.787180 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3fd3006f-9013-42cd-9bd6-335a07c91b04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:58.788580 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3fd3006f-9013-42cd-9bd6-335a07c91b04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:29:58.790768 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1129/2265] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:29:58 2022] DELETE /identity/v3/projects/a63bc70ae2fa4504a749058ad94e3b0d => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:29:58.795831 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09318155-58d9-4b1c-b31f-cb31436aa003 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:58.796190 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09318155-58d9-4b1c-b31f-cb31436aa003 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:58.796533 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09318155-58d9-4b1c-b31f-cb31436aa003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:58.810804 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1137/2266] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:58.811197 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557e855-1151-4c2d-9684-5bd6d5581af6 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:58.820597 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557e855-1151-4c2d-9684-5bd6d5581af6 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:58.822095 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-09318155-58d9-4b1c-b31f-cb31436aa003 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:58.823933 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-09318155-58d9-4b1c-b31f-cb31436aa003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:58.826322 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68127ee4-2685-4cb1-87d1-85bcae7dbc4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:58.827153 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68127ee4-2685-4cb1-87d1-85bcae7dbc4f admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:58.827505 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68127ee4-2685-4cb1-87d1-85bcae7dbc4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:58.828529 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68127ee4-2685-4cb1-87d1-85bcae7dbc4f admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:58.828529 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68127ee4-2685-4cb1-87d1-85bcae7dbc4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:29:58.829633 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68127ee4-2685-4cb1-87d1-85bcae7dbc4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:58.830758 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68127ee4-2685-4cb1-87d1-85bcae7dbc4f admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:58.843915 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1130/2267] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:29:58.849285 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1138/2268] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:29:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:29:58.853183 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68127ee4-2685-4cb1-87d1-85bcae7dbc4f admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:58.853555 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab504f-2ad3-49ac-8945-2d7eccc8f672 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:58.853883 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab504f-2ad3-49ac-8945-2d7eccc8f672 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:58.854163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab504f-2ad3-49ac-8945-2d7eccc8f672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:58.864253 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68127ee4-2685-4cb1-87d1-85bcae7dbc4f admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:58.869329 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a88d5-f825-492a-a626-def0a9dc154f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:58.870012 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a88d5-f825-492a-a626-def0a9dc154f admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:58.870358 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a88d5-f825-492a-a626-def0a9dc154f admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:58.870825 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a88d5-f825-492a-a626-def0a9dc154f admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:58.871526 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607a88d5-f825-492a-a626-def0a9dc154f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:58.873516 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607a88d5-f825-492a-a626-def0a9dc154f admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:58.877741 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-24ab504f-2ad3-49ac-8945-2d7eccc8f672 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:58.879003 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-24ab504f-2ad3-49ac-8945-2d7eccc8f672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:58.891060 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1131/2269] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:29:58.895314 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab504f-2ad3-49ac-8945-2d7eccc8f672 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:58.910674 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1139/2270] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:29:58 2022] POST /identity/v3/projects => generated 365 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:29:58.914491 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a88d5-f825-492a-a626-def0a9dc154f admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:58.915299 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab504f-2ad3-49ac-8945-2d7eccc8f672 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:58.991065 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:59.002211 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da11577-8ef2-4290-a880-de81e54cce84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:59.005688 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da11577-8ef2-4290-a880-de81e54cce84 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:59.006351 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da11577-8ef2-4290-a880-de81e54cce84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:59.007201 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da11577-8ef2-4290-a880-de81e54cce84 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:59.008104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da11577-8ef2-4290-a880-de81e54cce84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:29:59.008481 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da11577-8ef2-4290-a880-de81e54cce84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:59.009286 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a88d5-f825-492a-a626-def0a9dc154f admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:59.011193 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da11577-8ef2-4290-a880-de81e54cce84 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:59.016511 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14632077-fdc2-4a35-a470-c20bf9a36ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:59.017286 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14632077-fdc2-4a35-a470-c20bf9a36ae2 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:59.017776 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14632077-fdc2-4a35-a470-c20bf9a36ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:59.018133 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14632077-fdc2-4a35-a470-c20bf9a36ae2 admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:59.018843 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14632077-fdc2-4a35-a470-c20bf9a36ae2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:59.021487 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14632077-fdc2-4a35-a470-c20bf9a36ae2 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:59.030133 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1132/2271] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:29:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:29:59.034538 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da11577-8ef2-4290-a880-de81e54cce84 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:59.049369 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da11577-8ef2-4290-a880-de81e54cce84 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:59.056913 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1140/2272] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:29:58 2022] POST /identity/v3/users => generated 283 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:29:59.059683 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f4cb4-e0cc-4b56-a969-4e4bd43713fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:59.060761 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f4cb4-e0cc-4b56-a969-4e4bd43713fa admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:59.061019 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14632077-fdc2-4a35-a470-c20bf9a36ae2 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:59.061125 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f4cb4-e0cc-4b56-a969-4e4bd43713fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:59.061534 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f4cb4-e0cc-4b56-a969-4e4bd43713fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:59.062842 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67f4cb4-e0cc-4b56-a969-4e4bd43713fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:59.065939 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67f4cb4-e0cc-4b56-a969-4e4bd43713fa admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:59.073371 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14632077-fdc2-4a35-a470-c20bf9a36ae2 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:59.079047 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c6d92-29f8-4cbc-99f9-0c372307a725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:59.079972 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c6d92-29f8-4cbc-99f9-0c372307a725 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:59.080259 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c6d92-29f8-4cbc-99f9-0c372307a725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:59.081906 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c6d92-29f8-4cbc-99f9-0c372307a725 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:59.082638 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c6d92-29f8-4cbc-99f9-0c372307a725 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:59.084517 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c6d92-29f8-4cbc-99f9-0c372307a725 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:59.099127 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1141/2273] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:29:59.103326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c6d92-29f8-4cbc-99f9-0c372307a725 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:59.114088 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1133/2274] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:29:59 2022] POST /identity/v3/projects => generated 371 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:29:59.116329 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c6d92-29f8-4cbc-99f9-0c372307a725 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:59.126406 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f4cb4-e0cc-4b56-a969-4e4bd43713fa admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:59.189531 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:59.232595 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f4cb4-e0cc-4b56-a969-4e4bd43713fa admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:59.240487 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f185f30-c0d0-4e97-ba32-8366c80abea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:59.240930 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf68cd4-fb51-4e55-af6f-6398fc2cd717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:59.241660 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf68cd4-fb51-4e55-af6f-6398fc2cd717 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:59.241987 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f185f30-c0d0-4e97-ba32-8366c80abea1 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:59.242111 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf68cd4-fb51-4e55-af6f-6398fc2cd717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:59.242525 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf68cd4-fb51-4e55-af6f-6398fc2cd717 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:59.243053 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f185f30-c0d0-4e97-ba32-8366c80abea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:59.243330 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf68cd4-fb51-4e55-af6f-6398fc2cd717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:59.243751 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f185f30-c0d0-4e97-ba32-8366c80abea1 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:59.245462 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f185f30-c0d0-4e97-ba32-8366c80abea1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:59.245663 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf68cd4-fb51-4e55-af6f-6398fc2cd717 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:59.248190 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f185f30-c0d0-4e97-ba32-8366c80abea1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:59.259184 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1142/2275] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:59 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:29:59.264235 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf68cd4-fb51-4e55-af6f-6398fc2cd717 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:59.278273 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf68cd4-fb51-4e55-af6f-6398fc2cd717 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:59.280555 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1134/2276] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:29:59 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:29:59.285288 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d571aa-98c2-4d3b-b8df-299817f14e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:59.286079 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d571aa-98c2-4d3b-b8df-299817f14e6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:59.286463 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d571aa-98c2-4d3b-b8df-299817f14e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:59.286842 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d571aa-98c2-4d3b-b8df-299817f14e6e admin admin] PATH_INFO: `/v3/projects/2aef7ad8f6b34253a0b2e4172462e659/users/469b63edd8da4aeb8f0d6372fa376362/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:59.291278 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f185f30-c0d0-4e97-ba32-8366c80abea1 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:59.307831 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f185f30-c0d0-4e97-ba32-8366c80abea1 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:59.313620 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6534ac-868b-464e-93d6-67706a8e2257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:59.314691 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6534ac-868b-464e-93d6-67706a8e2257 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:59.315317 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6534ac-868b-464e-93d6-67706a8e2257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:59.316569 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6534ac-868b-464e-93d6-67706a8e2257 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:59.317601 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6534ac-868b-464e-93d6-67706a8e2257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:59.320092 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6534ac-868b-464e-93d6-67706a8e2257 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:59.330764 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d571aa-98c2-4d3b-b8df-299817f14e6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aef7ad8f6b34253a0b2e4172462e659, user_id=469b63edd8da4aeb8f0d6372fa376362, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:59.334954 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1135/2277] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:59 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:29:59.336337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d571aa-98c2-4d3b-b8df-299817f14e6e admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:59.342043 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6534ac-868b-464e-93d6-67706a8e2257 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:59.366671 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1143/2278] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:29:59 2022] PUT /identity/v3/projects/2aef7ad8f6b34253a0b2e4172462e659/users/469b63edd8da4aeb8f0d6372fa376362/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:59.376673 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6534ac-868b-464e-93d6-67706a8e2257 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:59.389545 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907f0d1-483b-4ab4-83c5-1e695cbb68bb None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:59.389545 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907f0d1-483b-4ab4-83c5-1e695cbb68bb None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:59.389545 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907f0d1-483b-4ab4-83c5-1e695cbb68bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:59.426386 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-4907f0d1-483b-4ab4-83c5-1e695cbb68bb None None] MFA Rules not processed for user `469b63edd8da4aeb8f0d6372fa376362`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:59.427733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-4907f0d1-483b-4ab4-83c5-1e695cbb68bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:59.476824 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3d6e6-7cac-41e1-a22d-ed63c0ec5bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:59.477571 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3d6e6-7cac-41e1-a22d-ed63c0ec5bf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:59.477867 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3d6e6-7cac-41e1-a22d-ed63c0ec5bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:59.478143 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3d6e6-7cac-41e1-a22d-ed63c0ec5bf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:59.478930 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f3d6e6-7cac-41e1-a22d-ed63c0ec5bf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:59.480708 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f3d6e6-7cac-41e1-a22d-ed63c0ec5bf5 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:59.494751 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1136/2279] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:59 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:29:59.498234 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3d6e6-7cac-41e1-a22d-ed63c0ec5bf5 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:59.510923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3d6e6-7cac-41e1-a22d-ed63c0ec5bf5 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:59.518032 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d62341-e589-4c39-a2d3-3ee9a9d0399e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:59.518896 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d62341-e589-4c39-a2d3-3ee9a9d0399e admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:59.519466 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d62341-e589-4c39-a2d3-3ee9a9d0399e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:59.519466 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d62341-e589-4c39-a2d3-3ee9a9d0399e admin admin] PATH_INFO: `/v3/projects/9e40b654b93e47dbac97665bcf044e27/users/8de232fc480e485d86bc2f0c4548bfed/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:59.536121 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1144/2280] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:29:59.538619 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d62341-e589-4c39-a2d3-3ee9a9d0399e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e40b654b93e47dbac97665bcf044e27, user_id=8de232fc480e485d86bc2f0c4548bfed, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:59.540710 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907f0d1-483b-4ab4-83c5-1e695cbb68bb None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:59.542134 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d62341-e589-4c39-a2d3-3ee9a9d0399e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:59.554526 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907f0d1-483b-4ab4-83c5-1e695cbb68bb None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:59.557453 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1137/2281] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:29:59 2022] PUT /identity/v3/projects/9e40b654b93e47dbac97665bcf044e27/users/8de232fc480e485d86bc2f0c4548bfed/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:29:59.567465 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853a6b8-c76b-4070-883a-27a60fbc29a1 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:59.567750 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853a6b8-c76b-4070-883a-27a60fbc29a1 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:59.568027 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853a6b8-c76b-4070-883a-27a60fbc29a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:59.599238 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-5853a6b8-c76b-4070-883a-27a60fbc29a1 None None] MFA Rules not processed for user `8de232fc480e485d86bc2f0c4548bfed`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:29:59.600909 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5853a6b8-c76b-4070-883a-27a60fbc29a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:59.608450 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:59.626857 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87dee5-5fae-4440-bb03-ebb47d996a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:59.627561 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87dee5-5fae-4440-bb03-ebb47d996a1d admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:59.627903 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87dee5-5fae-4440-bb03-ebb47d996a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:59.628234 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87dee5-5fae-4440-bb03-ebb47d996a1d admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:59.629232 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a87dee5-5fae-4440-bb03-ebb47d996a1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:59.631549 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a87dee5-5fae-4440-bb03-ebb47d996a1d admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:59.676156 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1145/2282] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:29:59 2022] POST /identity/v3/projects => generated 365 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:29:59.680865 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87dee5-5fae-4440-bb03-ebb47d996a1d admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:59.694555 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1138/2283] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:29:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:29:59.704525 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5853a6b8-c76b-4070-883a-27a60fbc29a1 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:59.706186 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5853a6b8-c76b-4070-883a-27a60fbc29a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:29:59.709707 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:59.754202 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87dee5-5fae-4440-bb03-ebb47d996a1d admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:59.760474 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8217b4d-1c38-46b3-87db-ed85d6ac0236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:59.761360 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8217b4d-1c38-46b3-87db-ed85d6ac0236 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:59.761733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8217b4d-1c38-46b3-87db-ed85d6ac0236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:59.762060 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8217b4d-1c38-46b3-87db-ed85d6ac0236 admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:59.762845 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8217b4d-1c38-46b3-87db-ed85d6ac0236 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:59.765308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8217b4d-1c38-46b3-87db-ed85d6ac0236 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:59.776694 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5853a6b8-c76b-4070-883a-27a60fbc29a1 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:59.788337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1146/2284] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:29:59 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:29:59.792923 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8217b4d-1c38-46b3-87db-ed85d6ac0236 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:59.804137 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8217b4d-1c38-46b3-87db-ed85d6ac0236 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:59.809775 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fff9ab1-a4d5-4d40-9b97-c9ba3124ad81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:59.810566 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff9ab1-a4d5-4d40-9b97-c9ba3124ad81 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:59.811439 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff9ab1-a4d5-4d40-9b97-c9ba3124ad81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:59.811789 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff9ab1-a4d5-4d40-9b97-c9ba3124ad81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:59.813746 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fff9ab1-a4d5-4d40-9b97-c9ba3124ad81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:59.815423 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fff9ab1-a4d5-4d40-9b97-c9ba3124ad81 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:59.831525 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1147/2285] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:29:59.835722 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fff9ab1-a4d5-4d40-9b97-c9ba3124ad81 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:59.838887 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb3e16-e2f7-4d77-a153-176af85b8a21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:59.839504 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb3e16-e2f7-4d77-a153-176af85b8a21 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:59.839791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb3e16-e2f7-4d77-a153-176af85b8a21 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:59.840067 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb3e16-e2f7-4d77-a153-176af85b8a21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:59.845812 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fff9ab1-a4d5-4d40-9b97-c9ba3124ad81 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:59.851493 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb223e2-bdd4-4494-ae1f-4da995c08cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:59.852138 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb223e2-bdd4-4494-ae1f-4da995c08cdc admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:59.852418 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb223e2-bdd4-4494-ae1f-4da995c08cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:59.852714 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb223e2-bdd4-4494-ae1f-4da995c08cdc admin admin] PATH_INFO: `/v3/projects/7a39ee033e2a4d4dbacc38c9171f2c04/users/d16744df6075480ba3de0f087cf9c23f/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:59.873280 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb223e2-bdd4-4494-ae1f-4da995c08cdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a39ee033e2a4d4dbacc38c9171f2c04, user_id=d16744df6075480ba3de0f087cf9c23f, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:59.875947 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb223e2-bdd4-4494-ae1f-4da995c08cdc admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:59.891577 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1148/2286] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:29:59 2022] PUT /identity/v3/projects/7a39ee033e2a4d4dbacc38c9171f2c04/users/d16744df6075480ba3de0f087cf9c23f/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:29:59.895254 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb223e2-bdd4-4494-ae1f-4da995c08cdc admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:59.904137 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdb3e16-e2f7-4d77-a153-176af85b8a21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:59.905726 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdb3e16-e2f7-4d77-a153-176af85b8a21 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:59.914780 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1139/2287] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:29:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 20:29:59.940434 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb3e16-e2f7-4d77-a153-176af85b8a21 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:59.951476 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb3e16-e2f7-4d77-a153-176af85b8a21 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:59.956077 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a994ef4-6d38-49c4-9d36-bf13b07614e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:59.956694 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a994ef4-6d38-49c4-9d36-bf13b07614e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:59.956974 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a994ef4-6d38-49c4-9d36-bf13b07614e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:59.957266 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a994ef4-6d38-49c4-9d36-bf13b07614e1 admin admin] PATH_INFO: `/v3/users/8de232fc480e485d86bc2f0c4548bfed` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:59.958403 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a994ef4-6d38-49c4-9d36-bf13b07614e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8de232fc480e485d86bc2f0c4548bfed)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:59.959211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb223e2-bdd4-4494-ae1f-4da995c08cdc admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:59.960295 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a994ef4-6d38-49c4-9d36-bf13b07614e1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:59.964079 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d373f-d4dd-4dc2-8909-d708267bb3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:59.964749 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d373f-d4dd-4dc2-8909-d708267bb3c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:59.965036 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d373f-d4dd-4dc2-8909-d708267bb3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:59.965340 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d373f-d4dd-4dc2-8909-d708267bb3c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:29:59.965984 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331d373f-d4dd-4dc2-8909-d708267bb3c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:29:59.967575 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331d373f-d4dd-4dc2-8909-d708267bb3c4 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:29:59.973169 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:29:59.978069 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1149/2288] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:29:59 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:29:59.982293 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d373f-d4dd-4dc2-8909-d708267bb3c4 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:29:59.992912 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d373f-d4dd-4dc2-8909-d708267bb3c4 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:29:59.998742 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b30cab-6262-41ce-928a-3b9bdbc3a45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:29:59.999390 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b30cab-6262-41ce-928a-3b9bdbc3a45d admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:29:59.999678 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b30cab-6262-41ce-928a-3b9bdbc3a45d admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:29:59.999955 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b30cab-6262-41ce-928a-3b9bdbc3a45d admin admin] PATH_INFO: `/v3/projects/7a39ee033e2a4d4dbacc38c9171f2c04/users/d16744df6075480ba3de0f087cf9c23f/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:00.002019 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b30cab-6262-41ce-928a-3b9bdbc3a45d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a39ee033e2a4d4dbacc38c9171f2c04, user_id=d16744df6075480ba3de0f087cf9c23f, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:00.004831 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b30cab-6262-41ce-928a-3b9bdbc3a45d admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:00.012682 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3a994ef4-6d38-49c4-9d36-bf13b07614e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8de232fc480e485d86bc2f0c4548bfed'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:00.016167 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1150/2289] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:29:59 2022] PUT /identity/v3/projects/7a39ee033e2a4d4dbacc38c9171f2c04/users/d16744df6075480ba3de0f087cf9c23f/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:30:00.025951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3a994ef4-6d38-49c4-9d36-bf13b07614e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8de232fc480e485d86bc2f0c4548bfed'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:00.027836 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb0e1a-dd48-4c87-9332-884b693b583e None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:00.028117 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb0e1a-dd48-4c87-9332-884b693b583e None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:00.028420 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb0e1a-dd48-4c87-9332-884b693b583e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:00.032035 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3a994ef4-6d38-49c4-9d36-bf13b07614e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8de232fc480e485d86bc2f0c4548bfed'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:00.039976 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3a994ef4-6d38-49c4-9d36-bf13b07614e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8de232fc480e485d86bc2f0c4548bfed'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:00.040727 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3a994ef4-6d38-49c4-9d36-bf13b07614e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8de232fc480e485d86bc2f0c4548bfed'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:00.046959 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1140/2290] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:29:59 2022] DELETE /identity/v3/users/8de232fc480e485d86bc2f0c4548bfed => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:30:00.054596 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba54127-819a-442c-b313-8be0f198ff91 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:00.054884 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba54127-819a-442c-b313-8be0f198ff91 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:00.055298 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba54127-819a-442c-b313-8be0f198ff91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:00.057483 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-7deb0e1a-dd48-4c87-9332-884b693b583e None None] MFA Rules not processed for user `d16744df6075480ba3de0f087cf9c23f`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:30:00.058678 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7deb0e1a-dd48-4c87-9332-884b693b583e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:00.075024 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0ba54127-819a-442c-b313-8be0f198ff91 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:30:00.076211 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0ba54127-819a-442c-b313-8be0f198ff91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:00.113442 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:30:00.138634 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1151/2291] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:30:00 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:30:00.142444 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1141/2292] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:30:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:30:00.149462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb0e1a-dd48-4c87-9332-884b693b583e None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:00.151305 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba54127-819a-442c-b313-8be0f198ff91 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:00.151753 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7deb0e1a-dd48-4c87-9332-884b693b583e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:00.152971 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0ba54127-819a-442c-b313-8be0f198ff91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:00.214132 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba54127-819a-442c-b313-8be0f198ff91 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:00.219277 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174222e6-713c-472f-bc45-d1bcdd6ba1f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:00.220001 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174222e6-713c-472f-bc45-d1bcdd6ba1f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:00.220358 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174222e6-713c-472f-bc45-d1bcdd6ba1f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:00.221064 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174222e6-713c-472f-bc45-d1bcdd6ba1f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:00.221341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb0e1a-dd48-4c87-9332-884b693b583e None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:00.226106 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9801f012-c645-465d-80d8-b9ac2427bf1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:00.226716 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801f012-c645-465d-80d8-b9ac2427bf1c service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:00.226998 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801f012-c645-465d-80d8-b9ac2427bf1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:00.227274 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801f012-c645-465d-80d8-b9ac2427bf1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:00.231755 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174222e6-713c-472f-bc45-d1bcdd6ba1f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:00.233521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174222e6-713c-472f-bc45-d1bcdd6ba1f9 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:00.241018 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9801f012-c645-465d-80d8-b9ac2427bf1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:00.242509 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9801f012-c645-465d-80d8-b9ac2427bf1c service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:00.244821 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1142/2293] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:30:00 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:30:00.253094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1152/2294] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:30:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:30:00.330506 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174222e6-713c-472f-bc45-d1bcdd6ba1f9 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:00.340457 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174222e6-713c-472f-bc45-d1bcdd6ba1f9 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:00.344489 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57eda5-8640-4074-b958-88dfa92d6874 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:00.345278 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57eda5-8640-4074-b958-88dfa92d6874 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:00.345405 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57eda5-8640-4074-b958-88dfa92d6874 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:00.345664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57eda5-8640-4074-b958-88dfa92d6874 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:00.347799 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0a57eda5-8640-4074-b958-88dfa92d6874 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:00.381664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:30:00.416946 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a57eda5-8640-4074-b958-88dfa92d6874 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:00.418455 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a57eda5-8640-4074-b958-88dfa92d6874 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:00.431256 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1143/2295] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:30:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:30:00.590397 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9801f012-c645-465d-80d8-b9ac2427bf1c service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:00.591926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-9801f012-c645-465d-80d8-b9ac2427bf1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:00.604882 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9801f012-c645-465d-80d8-b9ac2427bf1c service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:00.610853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f086c6-61aa-4b76-9d10-76f58c0eb1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:00.612086 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f086c6-61aa-4b76-9d10-76f58c0eb1ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:00.612443 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f086c6-61aa-4b76-9d10-76f58c0eb1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:00.612949 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f086c6-61aa-4b76-9d10-76f58c0eb1ad admin admin] PATH_INFO: `/v3/projects/9e40b654b93e47dbac97665bcf044e27` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:00.614156 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f086c6-61aa-4b76-9d10-76f58c0eb1ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e40b654b93e47dbac97665bcf044e27)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:00.616484 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f086c6-61aa-4b76-9d10-76f58c0eb1ad admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:00.666797 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-30f086c6-61aa-4b76-9d10-76f58c0eb1ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e40b654b93e47dbac97665bcf044e27', 'request_id': 'req-30f086c6-61aa-4b76-9d10-76f58c0eb1ad'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:00.675186 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-30f086c6-61aa-4b76-9d10-76f58c0eb1ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e40b654b93e47dbac97665bcf044e27', 'request_id': 'req-30f086c6-61aa-4b76-9d10-76f58c0eb1ad'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:00.688300 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-30f086c6-61aa-4b76-9d10-76f58c0eb1ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e40b654b93e47dbac97665bcf044e27', 'request_id': 'req-30f086c6-61aa-4b76-9d10-76f58c0eb1ad'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:00.699682 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-30f086c6-61aa-4b76-9d10-76f58c0eb1ad admin admin] The token cache is being invalidate because project 9e40b654b93e47dbac97665bcf044e27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:30:00.699969 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-30f086c6-61aa-4b76-9d10-76f58c0eb1ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:00.700616 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-30f086c6-61aa-4b76-9d10-76f58c0eb1ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:00.702133 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1153/2296] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:30:00 2022] DELETE /identity/v3/projects/9e40b654b93e47dbac97665bcf044e27 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:30:00.706955 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b7d62-e8d3-4973-9352-61235b04381b None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:00.707453 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b7d62-e8d3-4973-9352-61235b04381b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:00.707549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b7d62-e8d3-4973-9352-61235b04381b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:00.850229 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-499b7d62-e8d3-4973-9352-61235b04381b None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:30:00.851451 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-499b7d62-e8d3-4973-9352-61235b04381b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:00.917900 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1144/2297] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:30:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:30:00.925276 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9d9c7-cfac-41c4-9941-26c32f585fd5 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:00.925646 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9d9c7-cfac-41c4-9941-26c32f585fd5 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:00.926564 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9d9c7-cfac-41c4-9941-26c32f585fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:00.945882 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-61f9d9c7-cfac-41c4-9941-26c32f585fd5 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:30:00.947190 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-61f9d9c7-cfac-41c4-9941-26c32f585fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:00.955226 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1154/2298] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:30:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:30:00.959729 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b7d62-e8d3-4973-9352-61235b04381b None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:00.984684 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b7d62-e8d3-4973-9352-61235b04381b None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:00.992561 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5a6b7-ea95-4b7d-86c2-8626efa54d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:00.993330 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5a6b7-ea95-4b7d-86c2-8626efa54d62 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:00.993706 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5a6b7-ea95-4b7d-86c2-8626efa54d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:00.994088 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5a6b7-ea95-4b7d-86c2-8626efa54d62 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:00.994869 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b5a6b7-ea95-4b7d-86c2-8626efa54d62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:30:00.995272 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b5a6b7-ea95-4b7d-86c2-8626efa54d62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:00.996963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b5a6b7-ea95-4b7d-86c2-8626efa54d62 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:01.014457 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1145/2299] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:30:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:30:01.015752 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f9d9c7-cfac-41c4-9941-26c32f585fd5 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:01.025533 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f9d9c7-cfac-41c4-9941-26c32f585fd5 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:01.031168 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15542cf8-ee7a-419e-9cc6-ded3772dc565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:01.031850 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15542cf8-ee7a-419e-9cc6-ded3772dc565 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:01.032204 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15542cf8-ee7a-419e-9cc6-ded3772dc565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:01.032687 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15542cf8-ee7a-419e-9cc6-ded3772dc565 admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:01.033338 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15542cf8-ee7a-419e-9cc6-ded3772dc565 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:01.035274 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15542cf8-ee7a-419e-9cc6-ded3772dc565 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:01.066206 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1155/2300] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:30:01 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:30:01.067367 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5a6b7-ea95-4b7d-86c2-8626efa54d62 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:01.078205 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5a6b7-ea95-4b7d-86c2-8626efa54d62 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:01.144740 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1a1c0-6d51-4a2d-a985-315e42658590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:01.145381 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1a1c0-6d51-4a2d-a985-315e42658590 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:01.145686 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1a1c0-6d51-4a2d-a985-315e42658590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:01.145955 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1a1c0-6d51-4a2d-a985-315e42658590 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:01.146574 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b1a1c0-6d51-4a2d-a985-315e42658590 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:01.148468 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b1a1c0-6d51-4a2d-a985-315e42658590 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:01.172939 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1146/2301] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:30:01 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:30:01.174416 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15542cf8-ee7a-419e-9cc6-ded3772dc565 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:01.188277 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15542cf8-ee7a-419e-9cc6-ded3772dc565 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:01.193211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1635a9-7d8e-476f-a41f-2ae09aa6d8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:01.193843 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1635a9-7d8e-476f-a41f-2ae09aa6d8f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:01.194120 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1635a9-7d8e-476f-a41f-2ae09aa6d8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:01.194427 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1635a9-7d8e-476f-a41f-2ae09aa6d8f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:01.195038 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1635a9-7d8e-476f-a41f-2ae09aa6d8f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:01.196625 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1635a9-7d8e-476f-a41f-2ae09aa6d8f7 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:01.207232 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1156/2302] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:30:01.211247 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1a1c0-6d51-4a2d-a985-315e42658590 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:01.222167 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1a1c0-6d51-4a2d-a985-315e42658590 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:01.227297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c07e0e6-81a0-425b-a2ea-0e5960b1b242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:01.227981 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07e0e6-81a0-425b-a2ea-0e5960b1b242 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:01.228325 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07e0e6-81a0-425b-a2ea-0e5960b1b242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:01.228663 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07e0e6-81a0-425b-a2ea-0e5960b1b242 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:01.229406 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c07e0e6-81a0-425b-a2ea-0e5960b1b242 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:01.231203 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c07e0e6-81a0-425b-a2ea-0e5960b1b242 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:01.242102 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1147/2303] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:30:01.246058 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1635a9-7d8e-476f-a41f-2ae09aa6d8f7 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:01.255948 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1635a9-7d8e-476f-a41f-2ae09aa6d8f7 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:01.261029 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892991bf-b1bd-41fe-b353-059648246e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:01.261700 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892991bf-b1bd-41fe-b353-059648246e1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:01.261978 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892991bf-b1bd-41fe-b353-059648246e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:01.262249 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892991bf-b1bd-41fe-b353-059648246e1f admin admin] PATH_INFO: `/v3/projects/da5cdb6c865f4345be2de638e6ded3c6/users/ec90aaf1a60746f69e5478987ab86bc1/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:01.277113 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892991bf-b1bd-41fe-b353-059648246e1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=da5cdb6c865f4345be2de638e6ded3c6, user_id=ec90aaf1a60746f69e5478987ab86bc1, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:01.279839 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892991bf-b1bd-41fe-b353-059648246e1f admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:01.294107 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1157/2304] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:30:01 2022] PUT /identity/v3/projects/da5cdb6c865f4345be2de638e6ded3c6/users/ec90aaf1a60746f69e5478987ab86bc1/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:30:01.298999 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86440c3e-9e50-4665-baaf-7349df82c6f6 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:01.299294 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86440c3e-9e50-4665-baaf-7349df82c6f6 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:01.299565 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86440c3e-9e50-4665-baaf-7349df82c6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:01.318567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-86440c3e-9e50-4665-baaf-7349df82c6f6 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:30:01.319704 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-86440c3e-9e50-4665-baaf-7349df82c6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:01.353957 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:30:01.387790 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1148/2305] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:30:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:30:01.398655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892991bf-b1bd-41fe-b353-059648246e1f admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:01.401898 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-892991bf-b1bd-41fe-b353-059648246e1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:01.412090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:30:01.461808 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892991bf-b1bd-41fe-b353-059648246e1f admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:01.465988 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a88d846-13fd-49e5-a89f-29c6e44a3492 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:01.466921 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88d846-13fd-49e5-a89f-29c6e44a3492 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:01.466921 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88d846-13fd-49e5-a89f-29c6e44a3492 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:01.467181 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88d846-13fd-49e5-a89f-29c6e44a3492 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:01.476359 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a88d846-13fd-49e5-a89f-29c6e44a3492 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:01.477820 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a88d846-13fd-49e5-a89f-29c6e44a3492 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:01.488191 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1158/2306] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:30:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:30:04.601682 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86440c3e-9e50-4665-baaf-7349df82c6f6 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:04.603524 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-86440c3e-9e50-4665-baaf-7349df82c6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:04.625714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86440c3e-9e50-4665-baaf-7349df82c6f6 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:04.640752 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abedcaf3-bd61-4d3e-8cc2-52b9e1fc9b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:04.642337 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abedcaf3-bd61-4d3e-8cc2-52b9e1fc9b57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:04.642337 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abedcaf3-bd61-4d3e-8cc2-52b9e1fc9b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:04.642337 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abedcaf3-bd61-4d3e-8cc2-52b9e1fc9b57 admin admin] PATH_INFO: `/v3/users/469b63edd8da4aeb8f0d6372fa376362` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:04.644788 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abedcaf3-bd61-4d3e-8cc2-52b9e1fc9b57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=469b63edd8da4aeb8f0d6372fa376362)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:04.650084 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abedcaf3-bd61-4d3e-8cc2-52b9e1fc9b57 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:04.766946 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-abedcaf3-bd61-4d3e-8cc2-52b9e1fc9b57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '469b63edd8da4aeb8f0d6372fa376362'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:04.783706 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-abedcaf3-bd61-4d3e-8cc2-52b9e1fc9b57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '469b63edd8da4aeb8f0d6372fa376362'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:04.810917 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-abedcaf3-bd61-4d3e-8cc2-52b9e1fc9b57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '469b63edd8da4aeb8f0d6372fa376362'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:04.841438 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-abedcaf3-bd61-4d3e-8cc2-52b9e1fc9b57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '469b63edd8da4aeb8f0d6372fa376362'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:04.842310 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-abedcaf3-bd61-4d3e-8cc2-52b9e1fc9b57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '469b63edd8da4aeb8f0d6372fa376362'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:04.852336 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1149/2307] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:30:04 2022] DELETE /identity/v3/users/469b63edd8da4aeb8f0d6372fa376362 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:30:04.854588 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916bd69-1cab-4eed-a304-6f8496274201 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:04.855469 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916bd69-1cab-4eed-a304-6f8496274201 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:04.855902 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916bd69-1cab-4eed-a304-6f8496274201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:04.899545 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-0916bd69-1cab-4eed-a304-6f8496274201 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:30:04.901363 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-0916bd69-1cab-4eed-a304-6f8496274201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:05.021696 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1159/2308] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:30:05.028795 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abedcaf3-bd61-4d3e-8cc2-52b9e1fc9b57 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:05.030403 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-abedcaf3-bd61-4d3e-8cc2-52b9e1fc9b57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:05.146728 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abedcaf3-bd61-4d3e-8cc2-52b9e1fc9b57 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:05.158139 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e56fbb-f92c-4231-b4b2-85ca2b07da48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:05.163429 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e56fbb-f92c-4231-b4b2-85ca2b07da48 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:05.163696 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e56fbb-f92c-4231-b4b2-85ca2b07da48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:05.163942 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e56fbb-f92c-4231-b4b2-85ca2b07da48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:05.197472 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e56fbb-f92c-4231-b4b2-85ca2b07da48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:05.197472 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e56fbb-f92c-4231-b4b2-85ca2b07da48 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:05.222433 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1150/2309] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:30:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:30:06.095363 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916bd69-1cab-4eed-a304-6f8496274201 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:06.097510 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-0916bd69-1cab-4eed-a304-6f8496274201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:06.141807 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916bd69-1cab-4eed-a304-6f8496274201 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:06.154829 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0754a3-ada1-4308-b390-fdbe54ba968f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:06.155496 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0754a3-ada1-4308-b390-fdbe54ba968f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:06.156007 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0754a3-ada1-4308-b390-fdbe54ba968f admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:06.156282 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0754a3-ada1-4308-b390-fdbe54ba968f admin admin] PATH_INFO: `/v3/projects/2aef7ad8f6b34253a0b2e4172462e659` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:06.160939 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0754a3-ada1-4308-b390-fdbe54ba968f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aef7ad8f6b34253a0b2e4172462e659)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:06.167279 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0754a3-ada1-4308-b390-fdbe54ba968f admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:06.299056 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-1d0754a3-ada1-4308-b390-fdbe54ba968f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aef7ad8f6b34253a0b2e4172462e659', 'request_id': 'req-1d0754a3-ada1-4308-b390-fdbe54ba968f'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:06.310404 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-1d0754a3-ada1-4308-b390-fdbe54ba968f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aef7ad8f6b34253a0b2e4172462e659', 'request_id': 'req-1d0754a3-ada1-4308-b390-fdbe54ba968f'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:06.325391 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-1d0754a3-ada1-4308-b390-fdbe54ba968f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aef7ad8f6b34253a0b2e4172462e659', 'request_id': 'req-1d0754a3-ada1-4308-b390-fdbe54ba968f'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:06.340100 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-1d0754a3-ada1-4308-b390-fdbe54ba968f admin admin] The token cache is being invalidate because project 2aef7ad8f6b34253a0b2e4172462e659 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:30:06.340754 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-1d0754a3-ada1-4308-b390-fdbe54ba968f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:06.341051 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-1d0754a3-ada1-4308-b390-fdbe54ba968f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:06.342928 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1160/2310] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:30:06 2022] DELETE /identity/v3/projects/2aef7ad8f6b34253a0b2e4172462e659 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:30:06.347661 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e56fbb-f92c-4231-b4b2-85ca2b07da48 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:06.364238 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e56fbb-f92c-4231-b4b2-85ca2b07da48 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:06.446577 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:30:06.459344 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e874f4-8b80-4f81-a83e-5b424003681b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:06.460350 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e874f4-8b80-4f81-a83e-5b424003681b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:06.460801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e874f4-8b80-4f81-a83e-5b424003681b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:06.461706 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e874f4-8b80-4f81-a83e-5b424003681b admin admin] PATH_INFO: `/v3/users/d16744df6075480ba3de0f087cf9c23f` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:06.463467 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e874f4-8b80-4f81-a83e-5b424003681b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d16744df6075480ba3de0f087cf9c23f)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:06.466104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e874f4-8b80-4f81-a83e-5b424003681b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:06.600427 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e1e874f4-8b80-4f81-a83e-5b424003681b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd16744df6075480ba3de0f087cf9c23f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:06.609324 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e1e874f4-8b80-4f81-a83e-5b424003681b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd16744df6075480ba3de0f087cf9c23f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:06.633321 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e1e874f4-8b80-4f81-a83e-5b424003681b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd16744df6075480ba3de0f087cf9c23f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:06.643674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e1e874f4-8b80-4f81-a83e-5b424003681b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd16744df6075480ba3de0f087cf9c23f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:06.644777 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e1e874f4-8b80-4f81-a83e-5b424003681b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd16744df6075480ba3de0f087cf9c23f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:06.653213 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1151/2311] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:30:06 2022] DELETE /identity/v3/users/d16744df6075480ba3de0f087cf9c23f => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:30:07.171438 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0754a3-ada1-4308-b390-fdbe54ba968f admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:07.173998 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-1d0754a3-ada1-4308-b390-fdbe54ba968f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:07.190496 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0754a3-ada1-4308-b390-fdbe54ba968f admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:07.252890 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:30:07.295763 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704bc2a8-ea07-4871-ab5e-622c93128cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:07.296413 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704bc2a8-ea07-4871-ab5e-622c93128cc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:07.296694 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704bc2a8-ea07-4871-ab5e-622c93128cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:07.296972 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704bc2a8-ea07-4871-ab5e-622c93128cc4 admin admin] PATH_INFO: `/v3/projects/7a39ee033e2a4d4dbacc38c9171f2c04` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:07.299504 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704bc2a8-ea07-4871-ab5e-622c93128cc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a39ee033e2a4d4dbacc38c9171f2c04)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:07.302023 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704bc2a8-ea07-4871-ab5e-622c93128cc4 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:07.390065 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-704bc2a8-ea07-4871-ab5e-622c93128cc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a39ee033e2a4d4dbacc38c9171f2c04', 'request_id': 'req-704bc2a8-ea07-4871-ab5e-622c93128cc4'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:07.409756 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-704bc2a8-ea07-4871-ab5e-622c93128cc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a39ee033e2a4d4dbacc38c9171f2c04', 'request_id': 'req-704bc2a8-ea07-4871-ab5e-622c93128cc4'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:07.425978 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-704bc2a8-ea07-4871-ab5e-622c93128cc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a39ee033e2a4d4dbacc38c9171f2c04', 'request_id': 'req-704bc2a8-ea07-4871-ab5e-622c93128cc4'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:07.444936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-704bc2a8-ea07-4871-ab5e-622c93128cc4 admin admin] The token cache is being invalidate because project 7a39ee033e2a4d4dbacc38c9171f2c04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:30:07.445352 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-704bc2a8-ea07-4871-ab5e-622c93128cc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:07.448742 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-704bc2a8-ea07-4871-ab5e-622c93128cc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:07.451090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1161/2312] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:30:07 2022] DELETE /identity/v3/projects/7a39ee033e2a4d4dbacc38c9171f2c04 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:30:07.457194 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eefba1-49b8-41fc-a7a4-188d438c23e3 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:07.457720 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eefba1-49b8-41fc-a7a4-188d438c23e3 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:07.457802 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eefba1-49b8-41fc-a7a4-188d438c23e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:07.509485 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-84eefba1-49b8-41fc-a7a4-188d438c23e3 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:30:07.514798 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-84eefba1-49b8-41fc-a7a4-188d438c23e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:07.626190 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1152/2313] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:30:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:30:07.627285 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc497438-e3a5-4c15-9958-716490ed943c None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:07.627577 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc497438-e3a5-4c15-9958-716490ed943c None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:07.627880 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc497438-e3a5-4c15-9958-716490ed943c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:07.672267 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-cc497438-e3a5-4c15-9958-716490ed943c None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:30:07.672267 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-cc497438-e3a5-4c15-9958-716490ed943c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:07.693033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce220d8a-9726-4e07-a837-260cfe8959b8 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:07.693160 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1162/2314] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:30:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:30:07.694253 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce220d8a-9726-4e07-a837-260cfe8959b8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:07.694536 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce220d8a-9726-4e07-a837-260cfe8959b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:07.701570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc497438-e3a5-4c15-9958-716490ed943c None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:07.711337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc497438-e3a5-4c15-9958-716490ed943c None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:07.718900 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ad002-ed4d-4da6-a48d-b6e1d03778d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:07.719715 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ad002-ed4d-4da6-a48d-b6e1d03778d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:07.720348 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ad002-ed4d-4da6-a48d-b6e1d03778d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:07.720847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ad002-ed4d-4da6-a48d-b6e1d03778d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:07.721629 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ad002-ed4d-4da6-a48d-b6e1d03778d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:30:07.722007 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ad002-ed4d-4da6-a48d-b6e1d03778d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:07.723877 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ad002-ed4d-4da6-a48d-b6e1d03778d3 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:07.735350 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ce220d8a-9726-4e07-a837-260cfe8959b8 None None] MFA Rules not processed for user `ec90aaf1a60746f69e5478987ab86bc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:30:07.736824 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ce220d8a-9726-4e07-a837-260cfe8959b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:07.739977 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1163/2315] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:30:07.745062 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ad002-ed4d-4da6-a48d-b6e1d03778d3 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:07.755102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ad002-ed4d-4da6-a48d-b6e1d03778d3 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:07.760623 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e07dd-8c28-41c4-a3f8-f5cd45de1d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:07.761239 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e07dd-8c28-41c4-a3f8-f5cd45de1d37 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:07.761633 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e07dd-8c28-41c4-a3f8-f5cd45de1d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:07.761903 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e07dd-8c28-41c4-a3f8-f5cd45de1d37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:07.762919 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30e07dd-8c28-41c4-a3f8-f5cd45de1d37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:07.764836 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30e07dd-8c28-41c4-a3f8-f5cd45de1d37 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:07.802215 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1164/2316] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:30:07 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:30:07.803440 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e07dd-8c28-41c4-a3f8-f5cd45de1d37 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:07.813378 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1153/2317] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:30:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:30:07.817637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce220d8a-9726-4e07-a837-260cfe8959b8 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:07.819189 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ce220d8a-9726-4e07-a837-260cfe8959b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:07.827995 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:30:07.834176 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce220d8a-9726-4e07-a837-260cfe8959b8 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:07.877255 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e07dd-8c28-41c4-a3f8-f5cd45de1d37 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:07.882150 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c28c3e5-28d3-4d50-889d-d998cfeac750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:07.882773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28c3e5-28d3-4d50-889d-d998cfeac750 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:07.883049 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28c3e5-28d3-4d50-889d-d998cfeac750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:07.883373 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28c3e5-28d3-4d50-889d-d998cfeac750 admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:07.884014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c28c3e5-28d3-4d50-889d-d998cfeac750 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:07.886866 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c28c3e5-28d3-4d50-889d-d998cfeac750 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:07.939164 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b2f362-e427-46b6-8f05-1a1616fee08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:07.939739 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1165/2318] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:30:07 2022] POST /identity/v3/users => generated 291 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:30:07.939877 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2f362-e427-46b6-8f05-1a1616fee08e admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:07.940045 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2f362-e427-46b6-8f05-1a1616fee08e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:07.940319 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2f362-e427-46b6-8f05-1a1616fee08e admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:07.941013 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b2f362-e427-46b6-8f05-1a1616fee08e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:07.943116 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b2f362-e427-46b6-8f05-1a1616fee08e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:07.943983 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c28c3e5-28d3-4d50-889d-d998cfeac750 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:07.956618 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c28c3e5-28d3-4d50-889d-d998cfeac750 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:07.961401 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac818852-ec35-4294-ad82-db016e85ec4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:07.962018 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac818852-ec35-4294-ad82-db016e85ec4a admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:07.962301 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac818852-ec35-4294-ad82-db016e85ec4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:07.962753 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac818852-ec35-4294-ad82-db016e85ec4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:07.963376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac818852-ec35-4294-ad82-db016e85ec4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:07.965086 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac818852-ec35-4294-ad82-db016e85ec4a admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:07.973945 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:30:07.980866 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1154/2319] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:30:07 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:30:07.982354 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b2f362-e427-46b6-8f05-1a1616fee08e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:07.990185 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1166/2320] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:30:07.991347 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac818852-ec35-4294-ad82-db016e85ec4a admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:08.019670 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac818852-ec35-4294-ad82-db016e85ec4a admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:08.087697 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b2f362-e427-46b6-8f05-1a1616fee08e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:08.103118 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c2834-7113-4553-81f2-4cea964e7d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:08.103788 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c2834-7113-4553-81f2-4cea964e7d9d admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:08.104071 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c2834-7113-4553-81f2-4cea964e7d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:08.104346 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c2834-7113-4553-81f2-4cea964e7d9d admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:08.106163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c2834-7113-4553-81f2-4cea964e7d9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:08.108102 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c2834-7113-4553-81f2-4cea964e7d9d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:08.147417 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1155/2321] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:30:07 2022] POST /identity/v3/users => generated 299 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:30:08.155995 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c2834-7113-4553-81f2-4cea964e7d9d admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:08.158885 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38909489-ccfa-4c86-8057-e0c31b465df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:08.159625 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38909489-ccfa-4c86-8057-e0c31b465df2 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:08.160010 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38909489-ccfa-4c86-8057-e0c31b465df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:08.160374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38909489-ccfa-4c86-8057-e0c31b465df2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:08.161120 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38909489-ccfa-4c86-8057-e0c31b465df2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:08.163161 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38909489-ccfa-4c86-8057-e0c31b465df2 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:08.166590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c2834-7113-4553-81f2-4cea964e7d9d admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:08.172258 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02feab10-8e54-4bab-a5f0-ff4813dfcaea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:08.172925 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02feab10-8e54-4bab-a5f0-ff4813dfcaea admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:08.173213 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02feab10-8e54-4bab-a5f0-ff4813dfcaea admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:08.173568 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02feab10-8e54-4bab-a5f0-ff4813dfcaea admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:08.174148 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02feab10-8e54-4bab-a5f0-ff4813dfcaea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:08.175238 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1167/2322] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:30:08.175878 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02feab10-8e54-4bab-a5f0-ff4813dfcaea admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:08.179793 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38909489-ccfa-4c86-8057-e0c31b465df2 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:08.189441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1156/2323] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:30:08.193506 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02feab10-8e54-4bab-a5f0-ff4813dfcaea admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:08.195066 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38909489-ccfa-4c86-8057-e0c31b465df2 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:08.202879 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd760c-ff6a-4cf8-a304-bee63f43303f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:08.203833 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd760c-ff6a-4cf8-a304-bee63f43303f admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:08.204121 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd760c-ff6a-4cf8-a304-bee63f43303f admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:08.204395 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd760c-ff6a-4cf8-a304-bee63f43303f admin admin] PATH_INFO: `/v3/projects/622a8537dfdd4bf3a5d150ad778027a6/users/fedd69b51267403a979add58708b69ea/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:08.209042 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02feab10-8e54-4bab-a5f0-ff4813dfcaea admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:08.222632 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deba6cc-5a35-4604-9c66-c69296ad9b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:08.223546 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deba6cc-5a35-4604-9c66-c69296ad9b6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:08.224151 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deba6cc-5a35-4604-9c66-c69296ad9b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:08.224603 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deba6cc-5a35-4604-9c66-c69296ad9b6e admin admin] PATH_INFO: `/v3/projects/33294a040dca4909a3cbeda469245c75/users/9a27aba7d18942fe80504abd4006f5e6/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:08.240020 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd760c-ff6a-4cf8-a304-bee63f43303f admin admin] RBAC: Authorizing `identity:create_grant(project_id=622a8537dfdd4bf3a5d150ad778027a6, user_id=fedd69b51267403a979add58708b69ea, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:08.245925 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd760c-ff6a-4cf8-a304-bee63f43303f admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:08.254788 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deba6cc-5a35-4604-9c66-c69296ad9b6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=33294a040dca4909a3cbeda469245c75, user_id=9a27aba7d18942fe80504abd4006f5e6, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:08.258746 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deba6cc-5a35-4604-9c66-c69296ad9b6e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:08.282506 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1157/2324] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:30:08 2022] PUT /identity/v3/projects/33294a040dca4909a3cbeda469245c75/users/9a27aba7d18942fe80504abd4006f5e6/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:30:08.282703 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1168/2325] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:30:08 2022] PUT /identity/v3/projects/622a8537dfdd4bf3a5d150ad778027a6/users/fedd69b51267403a979add58708b69ea/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:30:08.283556 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deba6cc-5a35-4604-9c66-c69296ad9b6e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:08.294318 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b8e53-79e8-45ee-bbaf-fdfe4f88f3e7 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:08.294596 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b8e53-79e8-45ee-bbaf-fdfe4f88f3e7 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:08.294880 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b8e53-79e8-45ee-bbaf-fdfe4f88f3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:08.338638 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-818b8e53-79e8-45ee-bbaf-fdfe4f88f3e7 None None] MFA Rules not processed for user `fedd69b51267403a979add58708b69ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:30:08.340860 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-818b8e53-79e8-45ee-bbaf-fdfe4f88f3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:08.356767 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:30:08.386825 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deba6cc-5a35-4604-9c66-c69296ad9b6e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:08.393840 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e040e1-9079-4828-abde-e460d72bc910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:08.394688 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e040e1-9079-4828-abde-e460d72bc910 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:08.395857 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e040e1-9079-4828-abde-e460d72bc910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:08.396243 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e040e1-9079-4828-abde-e460d72bc910 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:08.396967 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e040e1-9079-4828-abde-e460d72bc910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:08.398695 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e040e1-9079-4828-abde-e460d72bc910 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:08.410278 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1158/2326] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:30:08.414654 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e040e1-9079-4828-abde-e460d72bc910 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:08.424585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e040e1-9079-4828-abde-e460d72bc910 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:08.433477 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc84aff0-70fd-49ea-8087-812f03f9de6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:08.434232 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84aff0-70fd-49ea-8087-812f03f9de6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:08.434578 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84aff0-70fd-49ea-8087-812f03f9de6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:08.434935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84aff0-70fd-49ea-8087-812f03f9de6e admin admin] PATH_INFO: `/v3/projects/33294a040dca4909a3cbeda469245c75/users/9a27aba7d18942fe80504abd4006f5e6/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:08.439871 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1169/2327] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:30:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:30:08.440036 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc84aff0-70fd-49ea-8087-812f03f9de6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=33294a040dca4909a3cbeda469245c75, user_id=9a27aba7d18942fe80504abd4006f5e6, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:08.441318 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc84aff0-70fd-49ea-8087-812f03f9de6e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:08.441441 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b8e53-79e8-45ee-bbaf-fdfe4f88f3e7 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:08.454510 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1159/2328] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:30:08 2022] PUT /identity/v3/projects/33294a040dca4909a3cbeda469245c75/users/9a27aba7d18942fe80504abd4006f5e6/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:30:08.457510 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b8e53-79e8-45ee-bbaf-fdfe4f88f3e7 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:08.532437 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be30b9-1749-4d34-991f-d1af07553a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:08.533064 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be30b9-1749-4d34-991f-d1af07553a23 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:08.533534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be30b9-1749-4d34-991f-d1af07553a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:08.533823 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be30b9-1749-4d34-991f-d1af07553a23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:08.534852 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87be30b9-1749-4d34-991f-d1af07553a23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:08.537192 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87be30b9-1749-4d34-991f-d1af07553a23 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:08.546958 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:30:08.595181 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1170/2329] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:30:08 2022] POST /identity/v3/projects => generated 383 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:30:08.598736 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc84aff0-70fd-49ea-8087-812f03f9de6e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:08.614003 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc84aff0-70fd-49ea-8087-812f03f9de6e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:08.695407 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd03f8-68ff-461a-aa78-e3289a5d6d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:08.696150 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd03f8-68ff-461a-aa78-e3289a5d6d16 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:08.696492 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd03f8-68ff-461a-aa78-e3289a5d6d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:08.697088 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd03f8-68ff-461a-aa78-e3289a5d6d16 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:08.697621 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd03f8-68ff-461a-aa78-e3289a5d6d16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:08.700032 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd03f8-68ff-461a-aa78-e3289a5d6d16 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:08.732468 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1160/2330] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:30:08 2022] POST /identity/v3/users => generated 298 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:30:08.733867 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be30b9-1749-4d34-991f-d1af07553a23 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:08.746950 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be30b9-1749-4d34-991f-d1af07553a23 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:08.753399 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0a4c9a-5281-416c-8147-fe3b64333486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:08.754000 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a4c9a-5281-416c-8147-fe3b64333486 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:08.754334 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a4c9a-5281-416c-8147-fe3b64333486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:08.754762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a4c9a-5281-416c-8147-fe3b64333486 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:08.755432 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0a4c9a-5281-416c-8147-fe3b64333486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:08.758371 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0a4c9a-5281-416c-8147-fe3b64333486 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:08.781151 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1171/2331] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:30:08.782236 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd03f8-68ff-461a-aa78-e3289a5d6d16 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:08.793940 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd03f8-68ff-461a-aa78-e3289a5d6d16 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:08.798879 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030a6cd-fd1e-4054-bd22-7722400ce38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:08.799806 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030a6cd-fd1e-4054-bd22-7722400ce38f admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:08.800359 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030a6cd-fd1e-4054-bd22-7722400ce38f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:08.800359 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030a6cd-fd1e-4054-bd22-7722400ce38f admin admin] PATH_INFO: `/v3/projects/bfbf71b100804985810af33821535e48/users/6a654d775b654a918b510a84b2c1faeb/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:08.821988 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a030a6cd-fd1e-4054-bd22-7722400ce38f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfbf71b100804985810af33821535e48, user_id=6a654d775b654a918b510a84b2c1faeb, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:08.824823 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a030a6cd-fd1e-4054-bd22-7722400ce38f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:08.840658 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1161/2332] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:30:08 2022] PUT /identity/v3/projects/bfbf71b100804985810af33821535e48/users/6a654d775b654a918b510a84b2c1faeb/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:30:08.844695 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0a4c9a-5281-416c-8147-fe3b64333486 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:08.856045 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0a4c9a-5281-416c-8147-fe3b64333486 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:08.927174 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38475dfa-32f1-4a20-ac87-62ed4b4e9fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:08.927783 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38475dfa-32f1-4a20-ac87-62ed4b4e9fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:08.928372 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38475dfa-32f1-4a20-ac87-62ed4b4e9fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:08.928652 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38475dfa-32f1-4a20-ac87-62ed4b4e9fa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:08.929350 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38475dfa-32f1-4a20-ac87-62ed4b4e9fa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:08.932037 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38475dfa-32f1-4a20-ac87-62ed4b4e9fa8 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:08.943886 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1172/2333] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:30:08.949709 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030a6cd-fd1e-4054-bd22-7722400ce38f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:08.963988 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030a6cd-fd1e-4054-bd22-7722400ce38f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:08.977745 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4064662a-860b-4403-a223-343c8218ec45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:08.979013 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064662a-860b-4403-a223-343c8218ec45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:08.979301 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064662a-860b-4403-a223-343c8218ec45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:08.979581 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064662a-860b-4403-a223-343c8218ec45 admin admin] PATH_INFO: `/v3/projects/bfbf71b100804985810af33821535e48/users/6a654d775b654a918b510a84b2c1faeb/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:08.994644 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4064662a-860b-4403-a223-343c8218ec45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfbf71b100804985810af33821535e48, user_id=6a654d775b654a918b510a84b2c1faeb, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:08.999504 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4064662a-860b-4403-a223-343c8218ec45 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:09.017124 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1162/2334] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:30:08 2022] PUT /identity/v3/projects/bfbf71b100804985810af33821535e48/users/6a654d775b654a918b510a84b2c1faeb/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:30:09.028071 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d71e0-455b-4c95-b033-7b17ac201c30 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:09.028439 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d71e0-455b-4c95-b033-7b17ac201c30 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:09.029286 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d71e0-455b-4c95-b033-7b17ac201c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:09.078404 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-3e7d71e0-455b-4c95-b033-7b17ac201c30 None None] MFA Rules not processed for user `6a654d775b654a918b510a84b2c1faeb`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:30:09.079991 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-3e7d71e0-455b-4c95-b033-7b17ac201c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:09.214932 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1173/2335] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:30:09 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:30:09.222521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4064662a-860b-4403-a223-343c8218ec45 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:09.224306 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4064662a-860b-4403-a223-343c8218ec45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:09.307708 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4064662a-860b-4403-a223-343c8218ec45 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:09.317136 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428584f-e2cc-4997-b9fb-23bd490f33c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:09.318120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428584f-e2cc-4997-b9fb-23bd490f33c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:09.319073 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428584f-e2cc-4997-b9fb-23bd490f33c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:09.320058 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428584f-e2cc-4997-b9fb-23bd490f33c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:09.341711 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a428584f-e2cc-4997-b9fb-23bd490f33c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:09.343161 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a428584f-e2cc-4997-b9fb-23bd490f33c0 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:09.359998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1163/2336] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:30:09 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:30:09.449629 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d71e0-455b-4c95-b033-7b17ac201c30 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:09.464348 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d71e0-455b-4c95-b033-7b17ac201c30 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:09.468362 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4edb6ec-f7f1-4ab3-875c-017e27467c7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:09.469829 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edb6ec-f7f1-4ab3-875c-017e27467c7b service cinder] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:09.470113 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edb6ec-f7f1-4ab3-875c-017e27467c7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:09.470389 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edb6ec-f7f1-4ab3-875c-017e27467c7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:09.569337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:30:09.574657 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4edb6ec-f7f1-4ab3-875c-017e27467c7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:09.577464 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4edb6ec-f7f1-4ab3-875c-017e27467c7b service cinder] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:09.603903 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1174/2337] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:30:09 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:30:09.942165 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ecbd9-f611-4f23-9d44-01b28ba3d224 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:09.942678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ecbd9-f611-4f23-9d44-01b28ba3d224 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:09.942897 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ecbd9-f611-4f23-9d44-01b28ba3d224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:09.987122 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-b81ecbd9-f611-4f23-9d44-01b28ba3d224 None None] MFA Rules not processed for user `cedaf156c5584d62beedde598c94f322`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:30:09.991168 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b81ecbd9-f611-4f23-9d44-01b28ba3d224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:10.044018 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:30:10.131487 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4edb6ec-f7f1-4ab3-875c-017e27467c7b service cinder] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:10.136850 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1164/2338] 10.222.0.26 () {58 vars in 1009 bytes} [Fri Sep 9 20:30:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:30:10.205327 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ecbd9-f611-4f23-9d44-01b28ba3d224 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:10.206992 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b81ecbd9-f611-4f23-9d44-01b28ba3d224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:10.342752 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4edb6ec-f7f1-4ab3-875c-017e27467c7b service cinder] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:10.355841 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e1a481-7c5e-42b0-82e6-e40ee5a7e2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:10.356514 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1a481-7c5e-42b0-82e6-e40ee5a7e2cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:10.356797 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1a481-7c5e-42b0-82e6-e40ee5a7e2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:10.357074 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1a481-7c5e-42b0-82e6-e40ee5a7e2cf admin admin] PATH_INFO: `/v3/users/fedd69b51267403a979add58708b69ea` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:10.361283 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e1a481-7c5e-42b0-82e6-e40ee5a7e2cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=fedd69b51267403a979add58708b69ea)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:10.365389 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e1a481-7c5e-42b0-82e6-e40ee5a7e2cf admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:10.373466 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ecbd9-f611-4f23-9d44-01b28ba3d224 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:10.386922 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f916967-6797-468a-9480-f046fa5ba5b9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb62d06f7e94e9295c54504ddabb1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:10.387826 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f916967-6797-468a-9480-f046fa5ba5b9 service swift] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:10.388116 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f916967-6797-468a-9480-f046fa5ba5b9 service swift] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:10.388394 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f916967-6797-468a-9480-f046fa5ba5b9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:10.403113 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f916967-6797-468a-9480-f046fa5ba5b9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:10.404749 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f916967-6797-468a-9480-f046fa5ba5b9 service swift] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:10.429841 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1165/2339] 10.222.0.26 () {60 vars in 1384 bytes} [Fri Sep 9 20:30:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 20:30:10.448262 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f916967-6797-468a-9480-f046fa5ba5b9 service swift] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:10.467185 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2f916967-6797-468a-9480-f046fa5ba5b9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:10.484006 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-b5e1a481-7c5e-42b0-82e6-e40ee5a7e2cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fedd69b51267403a979add58708b69ea'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:10.517279 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-b5e1a481-7c5e-42b0-82e6-e40ee5a7e2cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fedd69b51267403a979add58708b69ea'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:10.533053 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f916967-6797-468a-9480-f046fa5ba5b9 service swift] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:10.546444 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-b5e1a481-7c5e-42b0-82e6-e40ee5a7e2cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fedd69b51267403a979add58708b69ea'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:10.553664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe462db-92dd-4a29-83f9-b2edd3962d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:10.556018 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe462db-92dd-4a29-83f9-b2edd3962d6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:10.558710 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe462db-92dd-4a29-83f9-b2edd3962d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:10.559221 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe462db-92dd-4a29-83f9-b2edd3962d6e admin admin] PATH_INFO: `/v3/users/929aaf82cf4b42caaa615f6bed53519d` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:10.563695 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe462db-92dd-4a29-83f9-b2edd3962d6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=929aaf82cf4b42caaa615f6bed53519d)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:10.568691 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe462db-92dd-4a29-83f9-b2edd3962d6e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:10.569876 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-b5e1a481-7c5e-42b0-82e6-e40ee5a7e2cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fedd69b51267403a979add58708b69ea'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:10.570635 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-b5e1a481-7c5e-42b0-82e6-e40ee5a7e2cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fedd69b51267403a979add58708b69ea'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:10.583679 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1175/2340] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:30:10 2022] DELETE /identity/v3/users/fedd69b51267403a979add58708b69ea => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:30:10.589091 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77684986-7476-4d56-a6e9-583d5dc95747 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:10.589412 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77684986-7476-4d56-a6e9-583d5dc95747 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:10.589690 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77684986-7476-4d56-a6e9-583d5dc95747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:10.654487 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-77684986-7476-4d56-a6e9-583d5dc95747 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:30:10.664232 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-77684986-7476-4d56-a6e9-583d5dc95747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:10.784788 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-dbe462db-92dd-4a29-83f9-b2edd3962d6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '929aaf82cf4b42caaa615f6bed53519d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:10.806731 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1176/2341] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:30:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:30:10.813687 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-dbe462db-92dd-4a29-83f9-b2edd3962d6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '929aaf82cf4b42caaa615f6bed53519d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:10.848658 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-dbe462db-92dd-4a29-83f9-b2edd3962d6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '929aaf82cf4b42caaa615f6bed53519d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:10.864255 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77684986-7476-4d56-a6e9-583d5dc95747 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:10.866005 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-77684986-7476-4d56-a6e9-583d5dc95747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:10.870093 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-dbe462db-92dd-4a29-83f9-b2edd3962d6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '929aaf82cf4b42caaa615f6bed53519d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:10.871150 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-dbe462db-92dd-4a29-83f9-b2edd3962d6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '929aaf82cf4b42caaa615f6bed53519d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:10.888186 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1166/2342] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:30:10 2022] DELETE /identity/v3/users/929aaf82cf4b42caaa615f6bed53519d => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:30:10.890772 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b7e31-bedd-4ba9-a372-33859517f21b None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:10.891177 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b7e31-bedd-4ba9-a372-33859517f21b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:10.892068 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b7e31-bedd-4ba9-a372-33859517f21b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:10.906451 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:30:10.955300 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-cd5b7e31-bedd-4ba9-a372-33859517f21b None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:30:10.960660 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-cd5b7e31-bedd-4ba9-a372-33859517f21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:11.012584 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:30:11.119638 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77684986-7476-4d56-a6e9-583d5dc95747 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:11.147342 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1167/2343] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:30:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:30:11.160821 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b7e31-bedd-4ba9-a372-33859517f21b None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:11.166662 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-cd5b7e31-bedd-4ba9-a372-33859517f21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:11.295814 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a29da35-1cfe-4f3d-b335-21e2ffeb101b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:11.296692 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29da35-1cfe-4f3d-b335-21e2ffeb101b service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:11.297079 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29da35-1cfe-4f3d-b335-21e2ffeb101b service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:11.297489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29da35-1cfe-4f3d-b335-21e2ffeb101b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:11.308617 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-9a29da35-1cfe-4f3d-b335-21e2ffeb101b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:11.327242 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b7e31-bedd-4ba9-a372-33859517f21b None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:11.333708 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a29da35-1cfe-4f3d-b335-21e2ffeb101b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:11.335114 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef517ac-579b-4a4c-bf79-de9c472cc7fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:11.336091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef517ac-579b-4a4c-bf79-de9c472cc7fe service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:11.336537 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef517ac-579b-4a4c-bf79-de9c472cc7fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:11.336972 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef517ac-579b-4a4c-bf79-de9c472cc7fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:11.338057 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a29da35-1cfe-4f3d-b335-21e2ffeb101b service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:11.358025 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1177/2344] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:30:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 20:30:11.360175 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef517ac-579b-4a4c-bf79-de9c472cc7fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:11.361977 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef517ac-579b-4a4c-bf79-de9c472cc7fe service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:11.388718 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1168/2345] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:30:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 20:30:13.199301 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a29da35-1cfe-4f3d-b335-21e2ffeb101b service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:13.201973 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-9a29da35-1cfe-4f3d-b335-21e2ffeb101b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:13.311128 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a29da35-1cfe-4f3d-b335-21e2ffeb101b service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:13.360283 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ff873-c18a-4e70-a4ce-bd5c5a0d7e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:13.361075 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ff873-c18a-4e70-a4ce-bd5c5a0d7e4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:13.361383 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ff873-c18a-4e70-a4ce-bd5c5a0d7e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:13.361961 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ff873-c18a-4e70-a4ce-bd5c5a0d7e4b admin admin] PATH_INFO: `/v3/projects/51f7d6271f574a97a9277e584092b425` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:13.363473 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ff873-c18a-4e70-a4ce-bd5c5a0d7e4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=51f7d6271f574a97a9277e584092b425)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:13.367073 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ff873-c18a-4e70-a4ce-bd5c5a0d7e4b admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:13.447108 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef517ac-579b-4a4c-bf79-de9c472cc7fe service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:13.453615 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4ef517ac-579b-4a4c-bf79-de9c472cc7fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:13.498160 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef517ac-579b-4a4c-bf79-de9c472cc7fe service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:13.533538 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a6a0b0-206f-497a-85b6-cff327cfecdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:13.534242 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6a0b0-206f-497a-85b6-cff327cfecdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:13.534525 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6a0b0-206f-497a-85b6-cff327cfecdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:13.534795 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6a0b0-206f-497a-85b6-cff327cfecdb admin admin] PATH_INFO: `/v3/projects/622a8537dfdd4bf3a5d150ad778027a6` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:13.544742 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a6a0b0-206f-497a-85b6-cff327cfecdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=622a8537dfdd4bf3a5d150ad778027a6)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:13.547334 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a6a0b0-206f-497a-85b6-cff327cfecdb admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:13.595673 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-c87ff873-c18a-4e70-a4ce-bd5c5a0d7e4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51f7d6271f574a97a9277e584092b425', 'request_id': 'req-c87ff873-c18a-4e70-a4ce-bd5c5a0d7e4b'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:13.613937 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-c87ff873-c18a-4e70-a4ce-bd5c5a0d7e4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51f7d6271f574a97a9277e584092b425', 'request_id': 'req-c87ff873-c18a-4e70-a4ce-bd5c5a0d7e4b'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:13.664390 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-c87ff873-c18a-4e70-a4ce-bd5c5a0d7e4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51f7d6271f574a97a9277e584092b425', 'request_id': 'req-c87ff873-c18a-4e70-a4ce-bd5c5a0d7e4b'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:13.696149 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-c87ff873-c18a-4e70-a4ce-bd5c5a0d7e4b admin admin] The token cache is being invalidate because project 51f7d6271f574a97a9277e584092b425 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:30:13.697112 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-c87ff873-c18a-4e70-a4ce-bd5c5a0d7e4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:13.700471 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-c87ff873-c18a-4e70-a4ce-bd5c5a0d7e4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:13.702830 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1178/2346] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:30:13 2022] DELETE /identity/v3/projects/51f7d6271f574a97a9277e584092b425 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:30:13.780531 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-42a6a0b0-206f-497a-85b6-cff327cfecdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '622a8537dfdd4bf3a5d150ad778027a6', 'request_id': 'req-42a6a0b0-206f-497a-85b6-cff327cfecdb'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:13.814574 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-42a6a0b0-206f-497a-85b6-cff327cfecdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '622a8537dfdd4bf3a5d150ad778027a6', 'request_id': 'req-42a6a0b0-206f-497a-85b6-cff327cfecdb'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:13.841700 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-42a6a0b0-206f-497a-85b6-cff327cfecdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '622a8537dfdd4bf3a5d150ad778027a6', 'request_id': 'req-42a6a0b0-206f-497a-85b6-cff327cfecdb'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:13.899296 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-42a6a0b0-206f-497a-85b6-cff327cfecdb admin admin] The token cache is being invalidate because project 622a8537dfdd4bf3a5d150ad778027a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:30:13.899519 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-42a6a0b0-206f-497a-85b6-cff327cfecdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:13.900662 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-42a6a0b0-206f-497a-85b6-cff327cfecdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:13.904764 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1169/2347] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:30:13 2022] DELETE /identity/v3/projects/622a8537dfdd4bf3a5d150ad778027a6 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:30:13.909876 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ff873-c18a-4e70-a4ce-bd5c5a0d7e4b admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:13.911670 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-c87ff873-c18a-4e70-a4ce-bd5c5a0d7e4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:14.073690 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:30:14.103376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ff873-c18a-4e70-a4ce-bd5c5a0d7e4b admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:14.249461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d256392-9f5e-4070-a5a0-0be18d033989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:14.250243 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d256392-9f5e-4070-a5a0-0be18d033989 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:14.250634 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d256392-9f5e-4070-a5a0-0be18d033989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:14.251014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d256392-9f5e-4070-a5a0-0be18d033989 admin admin] PATH_INFO: `/v3/users/6a654d775b654a918b510a84b2c1faeb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:14.252377 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d256392-9f5e-4070-a5a0-0be18d033989 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a654d775b654a918b510a84b2c1faeb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:14.254701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d256392-9f5e-4070-a5a0-0be18d033989 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:14.432690 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6d256392-9f5e-4070-a5a0-0be18d033989 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a654d775b654a918b510a84b2c1faeb'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:14.543195 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6d256392-9f5e-4070-a5a0-0be18d033989 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a654d775b654a918b510a84b2c1faeb'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:14.671196 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6d256392-9f5e-4070-a5a0-0be18d033989 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a654d775b654a918b510a84b2c1faeb'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:14.704898 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6d256392-9f5e-4070-a5a0-0be18d033989 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a654d775b654a918b510a84b2c1faeb'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:14.710501 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6d256392-9f5e-4070-a5a0-0be18d033989 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a654d775b654a918b510a84b2c1faeb'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:14.778175 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1179/2348] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:30:13 2022] DELETE /identity/v3/users/6a654d775b654a918b510a84b2c1faeb => generated 0 bytes in 870 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:30:16.157686 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a6a0b0-206f-497a-85b6-cff327cfecdb admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:16.160068 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-42a6a0b0-206f-497a-85b6-cff327cfecdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:16.191600 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a6a0b0-206f-497a-85b6-cff327cfecdb admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:16.311257 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:30:16.333269 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e1531-ecb5-4cab-8e0f-2ed41e8c1da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:16.333824 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e1531-ecb5-4cab-8e0f-2ed41e8c1da3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:16.334295 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e1531-ecb5-4cab-8e0f-2ed41e8c1da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:16.334794 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e1531-ecb5-4cab-8e0f-2ed41e8c1da3 admin admin] PATH_INFO: `/v3/projects/bfbf71b100804985810af33821535e48` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:16.336091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e1531-ecb5-4cab-8e0f-2ed41e8c1da3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfbf71b100804985810af33821535e48)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:16.338281 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e1531-ecb5-4cab-8e0f-2ed41e8c1da3 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:16.549279 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-033e1531-ecb5-4cab-8e0f-2ed41e8c1da3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfbf71b100804985810af33821535e48', 'request_id': 'req-033e1531-ecb5-4cab-8e0f-2ed41e8c1da3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:16.578061 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-033e1531-ecb5-4cab-8e0f-2ed41e8c1da3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfbf71b100804985810af33821535e48', 'request_id': 'req-033e1531-ecb5-4cab-8e0f-2ed41e8c1da3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:16.606947 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-033e1531-ecb5-4cab-8e0f-2ed41e8c1da3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfbf71b100804985810af33821535e48', 'request_id': 'req-033e1531-ecb5-4cab-8e0f-2ed41e8c1da3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:16.643560 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-033e1531-ecb5-4cab-8e0f-2ed41e8c1da3 admin admin] The token cache is being invalidate because project bfbf71b100804985810af33821535e48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:30:16.644394 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-033e1531-ecb5-4cab-8e0f-2ed41e8c1da3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:16.645388 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-033e1531-ecb5-4cab-8e0f-2ed41e8c1da3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:16.647594 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1170/2349] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:30:16 2022] DELETE /identity/v3/projects/bfbf71b100804985810af33821535e48 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:30:16.653568 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab3fd1-1ff8-40e6-abb6-8aee43a1ebe7 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:16.654047 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab3fd1-1ff8-40e6-abb6-8aee43a1ebe7 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:16.654127 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab3fd1-1ff8-40e6-abb6-8aee43a1ebe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:16.718594 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-0fab3fd1-1ff8-40e6-abb6-8aee43a1ebe7 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:30:16.720926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-0fab3fd1-1ff8-40e6-abb6-8aee43a1ebe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:17.010495 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1180/2350] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:30:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:30:17.017972 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa1ed9-0ecc-4c53-bca7-4174aa39e33e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:17.018259 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa1ed9-0ecc-4c53-bca7-4174aa39e33e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:17.018536 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa1ed9-0ecc-4c53-bca7-4174aa39e33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:17.042886 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e5aa1ed9-0ecc-4c53-bca7-4174aa39e33e None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:30:17.043799 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e5aa1ed9-0ecc-4c53-bca7-4174aa39e33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:17.053146 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1171/2351] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:30:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:30:17.057593 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab3fd1-1ff8-40e6-abb6-8aee43a1ebe7 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:17.067937 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab3fd1-1ff8-40e6-abb6-8aee43a1ebe7 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:17.072871 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1df0aac-d5c4-4be3-ba16-50f834a2d078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:17.073735 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df0aac-d5c4-4be3-ba16-50f834a2d078 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:17.074125 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df0aac-d5c4-4be3-ba16-50f834a2d078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:17.074480 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df0aac-d5c4-4be3-ba16-50f834a2d078 admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:17.075231 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1df0aac-d5c4-4be3-ba16-50f834a2d078 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:30:17.075626 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1df0aac-d5c4-4be3-ba16-50f834a2d078 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:17.077363 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1df0aac-d5c4-4be3-ba16-50f834a2d078 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:17.090356 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1181/2352] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:30:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:30:17.094339 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa1ed9-0ecc-4c53-bca7-4174aa39e33e None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:17.106342 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa1ed9-0ecc-4c53-bca7-4174aa39e33e None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:17.120856 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ad18a-bd93-4f9c-a26b-aa36210bcd85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:17.121904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ad18a-bd93-4f9c-a26b-aa36210bcd85 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:17.122275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ad18a-bd93-4f9c-a26b-aa36210bcd85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:17.122640 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ad18a-bd93-4f9c-a26b-aa36210bcd85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:17.123556 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888ad18a-bd93-4f9c-a26b-aa36210bcd85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:17.126275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888ad18a-bd93-4f9c-a26b-aa36210bcd85 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:17.190719 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1172/2353] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:30:17 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:30:17.194997 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1df0aac-d5c4-4be3-ba16-50f834a2d078 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:17.208969 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1df0aac-d5c4-4be3-ba16-50f834a2d078 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:17.247241 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:30:17.337267 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6b6dc-dc5b-454b-a237-d6436e1ee59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:17.339819 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6b6dc-dc5b-454b-a237-d6436e1ee59b admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:17.340705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6b6dc-dc5b-454b-a237-d6436e1ee59b admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:17.341610 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6b6dc-dc5b-454b-a237-d6436e1ee59b admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:17.344368 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6b6dc-dc5b-454b-a237-d6436e1ee59b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:17.347517 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6b6dc-dc5b-454b-a237-d6436e1ee59b admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:17.383908 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1182/2354] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:30:17 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:30:17.386107 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ad18a-bd93-4f9c-a26b-aa36210bcd85 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:17.404273 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ad18a-bd93-4f9c-a26b-aa36210bcd85 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:17.416134 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e8dbb-a25c-4c77-b3ac-2d8d97b180a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:17.417725 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e8dbb-a25c-4c77-b3ac-2d8d97b180a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:17.418225 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e8dbb-a25c-4c77-b3ac-2d8d97b180a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:17.418702 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e8dbb-a25c-4c77-b3ac-2d8d97b180a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:17.419571 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e8dbb-a25c-4c77-b3ac-2d8d97b180a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:17.422096 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e8dbb-a25c-4c77-b3ac-2d8d97b180a8 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:17.438043 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1173/2355] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:30:17 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:30:17.441650 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6b6dc-dc5b-454b-a237-d6436e1ee59b admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:17.453989 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6b6dc-dc5b-454b-a237-d6436e1ee59b admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:17.460753 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73259212-449a-4889-9b47-55f753b0518a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:17.461782 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73259212-449a-4889-9b47-55f753b0518a admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:17.462300 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73259212-449a-4889-9b47-55f753b0518a admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:17.462855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73259212-449a-4889-9b47-55f753b0518a admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:17.463729 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73259212-449a-4889-9b47-55f753b0518a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:17.466507 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73259212-449a-4889-9b47-55f753b0518a admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:17.480548 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1183/2356] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:30:17 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:30:17.484365 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e8dbb-a25c-4c77-b3ac-2d8d97b180a8 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:17.525924 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e8dbb-a25c-4c77-b3ac-2d8d97b180a8 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:17.539030 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a655afb-f9e0-42da-9ed0-aaefb6a9801a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:17.539844 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a655afb-f9e0-42da-9ed0-aaefb6a9801a admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:17.542720 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a655afb-f9e0-42da-9ed0-aaefb6a9801a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:17.543122 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a655afb-f9e0-42da-9ed0-aaefb6a9801a admin admin] PATH_INFO: `/v3/projects/087796fe88654e079c9546b126df8f83/users/662b9dab25a347108debeaf2265da8d8/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:17.575253 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a655afb-f9e0-42da-9ed0-aaefb6a9801a admin admin] RBAC: Authorizing `identity:create_grant(project_id=087796fe88654e079c9546b126df8f83, user_id=662b9dab25a347108debeaf2265da8d8, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:17.578641 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a655afb-f9e0-42da-9ed0-aaefb6a9801a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:17.606390 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1174/2357] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:30:17 2022] PUT /identity/v3/projects/087796fe88654e079c9546b126df8f83/users/662b9dab25a347108debeaf2265da8d8/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:30:17.620728 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6a55b-d3f2-4535-8b26-6214532d783e None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:17.621006 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6a55b-d3f2-4535-8b26-6214532d783e None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:17.621714 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6a55b-d3f2-4535-8b26-6214532d783e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:17.709419 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-00a6a55b-d3f2-4535-8b26-6214532d783e None None] MFA Rules not processed for user `662b9dab25a347108debeaf2265da8d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:30:17.710963 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-00a6a55b-d3f2-4535-8b26-6214532d783e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:17.882892 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1184/2358] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:30:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:30:17.883887 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a655afb-f9e0-42da-9ed0-aaefb6a9801a admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:18.089445 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a655afb-f9e0-42da-9ed0-aaefb6a9801a admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:18.107373 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd335fd-fad5-4623-b0af-849f3755fd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:18.108297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd335fd-fad5-4623-b0af-849f3755fd2e admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:18.108736 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd335fd-fad5-4623-b0af-849f3755fd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:18.109164 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd335fd-fad5-4623-b0af-849f3755fd2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:18.112749 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd335fd-fad5-4623-b0af-849f3755fd2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:18.117798 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd335fd-fad5-4623-b0af-849f3755fd2e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:18.205513 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a6a55b-d3f2-4535-8b26-6214532d783e None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:18.207971 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1175/2359] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:30:17 2022] POST /identity/v3/projects => generated 385 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:30:18.223711 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a6a55b-d3f2-4535-8b26-6214532d783e None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:18.348214 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d8750-219e-4ab1-b727-22e6946ef3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:18.349529 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d8750-219e-4ab1-b727-22e6946ef3c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:18.350063 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d8750-219e-4ab1-b727-22e6946ef3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:18.350752 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d8750-219e-4ab1-b727-22e6946ef3c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:18.355793 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d8750-219e-4ab1-b727-22e6946ef3c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:18.359445 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d8750-219e-4ab1-b727-22e6946ef3c2 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:18.422385 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1185/2360] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:30:18 2022] POST /identity/v3/users => generated 293 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:30:18.426030 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd335fd-fad5-4623-b0af-849f3755fd2e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:18.439466 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:30:18.444965 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd335fd-fad5-4623-b0af-849f3755fd2e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:18.455243 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b574d4-2eb6-4fc8-9b6f-9418bb50376e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:18.456945 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b574d4-2eb6-4fc8-9b6f-9418bb50376e admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:18.458120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b574d4-2eb6-4fc8-9b6f-9418bb50376e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:18.458683 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b574d4-2eb6-4fc8-9b6f-9418bb50376e admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:18.459479 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b574d4-2eb6-4fc8-9b6f-9418bb50376e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:18.463949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b574d4-2eb6-4fc8-9b6f-9418bb50376e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:18.521796 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d8750-219e-4ab1-b727-22e6946ef3c2 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:18.537604 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1176/2361] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:30:18.542573 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d8750-219e-4ab1-b727-22e6946ef3c2 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:18.562330 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed62228-d67d-40cb-9e38-6df151628687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:18.562967 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed62228-d67d-40cb-9e38-6df151628687 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:18.567416 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed62228-d67d-40cb-9e38-6df151628687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:18.568073 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed62228-d67d-40cb-9e38-6df151628687 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:18.568805 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed62228-d67d-40cb-9e38-6df151628687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:18.575986 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed62228-d67d-40cb-9e38-6df151628687 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:18.590272 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:30:18.617987 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1186/2362] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:30:18.623077 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b574d4-2eb6-4fc8-9b6f-9418bb50376e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:18.639173 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b574d4-2eb6-4fc8-9b6f-9418bb50376e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:18.648026 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8001d1d-acad-4774-bd56-c758c1fecb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:18.648899 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8001d1d-acad-4774-bd56-c758c1fecb04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:18.649204 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8001d1d-acad-4774-bd56-c758c1fecb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:18.649554 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8001d1d-acad-4774-bd56-c758c1fecb04 admin admin] PATH_INFO: `/v3/projects/592bb0bed89644c6bf68571805944a71/users/612d311905d243f0bd0557ff5f65bd13/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:18.675860 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8001d1d-acad-4774-bd56-c758c1fecb04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=592bb0bed89644c6bf68571805944a71, user_id=612d311905d243f0bd0557ff5f65bd13, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:18.679413 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8001d1d-acad-4774-bd56-c758c1fecb04 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:18.706423 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1177/2363] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:30:18 2022] PUT /identity/v3/projects/592bb0bed89644c6bf68571805944a71/users/612d311905d243f0bd0557ff5f65bd13/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:30:18.722440 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91d9b4-6c73-4e12-86dd-016f69d3e8a4 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:18.722995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91d9b4-6c73-4e12-86dd-016f69d3e8a4 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:18.722995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91d9b4-6c73-4e12-86dd-016f69d3e8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:18.799911 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-5d91d9b4-6c73-4e12-86dd-016f69d3e8a4 None None] MFA Rules not processed for user `612d311905d243f0bd0557ff5f65bd13`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:30:18.803166 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-5d91d9b4-6c73-4e12-86dd-016f69d3e8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:18.865724 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e03cc-8efb-41d4-ab5c-c39f692c392a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:18.866541 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e03cc-8efb-41d4-ab5c-c39f692c392a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:18.866891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e03cc-8efb-41d4-ab5c-c39f692c392a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:18.908451 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-6e4e03cc-8efb-41d4-ab5c-c39f692c392a None None] MFA Rules not processed for user `9a27aba7d18942fe80504abd4006f5e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:30:18.910241 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6e4e03cc-8efb-41d4-ab5c-c39f692c392a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:18.967121 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1187/2364] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:30:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:30:18.967121 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91d9b4-6c73-4e12-86dd-016f69d3e8a4 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:19.053394 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1178/2365] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:30:18 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:30:19.067238 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e03cc-8efb-41d4-ab5c-c39f692c392a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:19.075380 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6e4e03cc-8efb-41d4-ab5c-c39f692c392a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:19.137044 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91d9b4-6c73-4e12-86dd-016f69d3e8a4 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:19.143460 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d2364-d2ae-4f4b-88f2-4cf0f7b8de35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:19.144094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d2364-d2ae-4f4b-88f2-4cf0f7b8de35 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:19.145122 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d2364-d2ae-4f4b-88f2-4cf0f7b8de35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:19.145452 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d2364-d2ae-4f4b-88f2-4cf0f7b8de35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:19.146090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d2364-d2ae-4f4b-88f2-4cf0f7b8de35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:19.148472 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d2364-d2ae-4f4b-88f2-4cf0f7b8de35 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:19.193944 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1188/2366] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:30:18 2022] POST /identity/v3/projects => generated 383 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:30:19.199457 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d2364-d2ae-4f4b-88f2-4cf0f7b8de35 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:19.291618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e03cc-8efb-41d4-ab5c-c39f692c392a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:19.338738 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d2364-d2ae-4f4b-88f2-4cf0f7b8de35 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:19.352891 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414cdce0-5e6b-4b4d-b158-cb5bb79aae65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:19.353639 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cdce0-5e6b-4b4d-b158-cb5bb79aae65 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:19.354026 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cdce0-5e6b-4b4d-b158-cb5bb79aae65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:19.354401 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cdce0-5e6b-4b4d-b158-cb5bb79aae65 admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:19.356426 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414cdce0-5e6b-4b4d-b158-cb5bb79aae65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:19.360854 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414cdce0-5e6b-4b4d-b158-cb5bb79aae65 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:19.372540 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e711e2ae-bc5b-4051-b745-5f186521377e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5298486310004b63b51116dbc747e31d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:19.373499 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e711e2ae-bc5b-4051-b745-5f186521377e service glance] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:19.373947 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e711e2ae-bc5b-4051-b745-5f186521377e service glance] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:19.374654 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e711e2ae-bc5b-4051-b745-5f186521377e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:19.377606 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e711e2ae-bc5b-4051-b745-5f186521377e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:19.392196 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:30:19.497396 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1189/2367] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:30:19 2022] POST /identity/v3/users => generated 298 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:30:19.498689 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414cdce0-5e6b-4b4d-b158-cb5bb79aae65 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:19.517479 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:30:19.540885 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e711e2ae-bc5b-4051-b745-5f186521377e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:19.542424 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e711e2ae-bc5b-4051-b745-5f186521377e service glance] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:19.547885 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414cdce0-5e6b-4b4d-b158-cb5bb79aae65 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:19.563939 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1179/2368] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:30:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:30:19.591193 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c0aa0-4720-4922-8477-233bb5ae7053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:19.596842 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c0aa0-4720-4922-8477-233bb5ae7053 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:19.597461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c0aa0-4720-4922-8477-233bb5ae7053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:19.599498 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c0aa0-4720-4922-8477-233bb5ae7053 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:19.600421 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65c0aa0-4720-4922-8477-233bb5ae7053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:19.602355 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65c0aa0-4720-4922-8477-233bb5ae7053 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:19.627885 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1190/2369] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:30:19.632223 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e711e2ae-bc5b-4051-b745-5f186521377e service glance] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:19.649932 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e711e2ae-bc5b-4051-b745-5f186521377e service glance] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:19.662329 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feeb296-6596-4251-89c7-2a5a00b3b618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:19.663548 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feeb296-6596-4251-89c7-2a5a00b3b618 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:19.664062 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feeb296-6596-4251-89c7-2a5a00b3b618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:19.664586 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feeb296-6596-4251-89c7-2a5a00b3b618 admin admin] PATH_INFO: `/v3/projects/b8022c2f5f8845a7b96f96b9ea13ec46/users/c3c2ed10924947e6b355783ee3be122b/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:19.709027 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feeb296-6596-4251-89c7-2a5a00b3b618 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8022c2f5f8845a7b96f96b9ea13ec46, user_id=c3c2ed10924947e6b355783ee3be122b, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:19.713202 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feeb296-6596-4251-89c7-2a5a00b3b618 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:19.749591 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1180/2370] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:30:19 2022] PUT /identity/v3/projects/b8022c2f5f8845a7b96f96b9ea13ec46/users/c3c2ed10924947e6b355783ee3be122b/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:30:19.749737 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c0aa0-4720-4922-8477-233bb5ae7053 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:19.764247 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c0aa0-4720-4922-8477-233bb5ae7053 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:19.948876 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001e175-bf86-497d-ab95-8773c2a94aa3 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:19.949158 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001e175-bf86-497d-ab95-8773c2a94aa3 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:19.950178 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001e175-bf86-497d-ab95-8773c2a94aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:19.983072 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d6c49-fb94-4aa8-bef4-602f1d5dffee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:19.984039 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d6c49-fb94-4aa8-bef4-602f1d5dffee admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:19.984545 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d6c49-fb94-4aa8-bef4-602f1d5dffee admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:19.984904 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d6c49-fb94-4aa8-bef4-602f1d5dffee admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:19.985817 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d6c49-fb94-4aa8-bef4-602f1d5dffee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:19.987672 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d6c49-fb94-4aa8-bef4-602f1d5dffee admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:20.017416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0001e175-bf86-497d-ab95-8773c2a94aa3 None None] MFA Rules not processed for user `a95746e7a84541199b43138b1a348386`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:30:20.022327 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0001e175-bf86-497d-ab95-8773c2a94aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:20.025960 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1191/2371] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:30:20.030286 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d6c49-fb94-4aa8-bef4-602f1d5dffee admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:20.051225 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d6c49-fb94-4aa8-bef4-602f1d5dffee admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:20.056839 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92e3a5-80f3-4f01-b1f8-615038840117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:20.057847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92e3a5-80f3-4f01-b1f8-615038840117 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:20.058384 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92e3a5-80f3-4f01-b1f8-615038840117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:20.059235 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92e3a5-80f3-4f01-b1f8-615038840117 admin admin] PATH_INFO: `/v3/projects/b8022c2f5f8845a7b96f96b9ea13ec46/users/c3c2ed10924947e6b355783ee3be122b/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:20.061827 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b92e3a5-80f3-4f01-b1f8-615038840117 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8022c2f5f8845a7b96f96b9ea13ec46, user_id=c3c2ed10924947e6b355783ee3be122b, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:20.064835 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b92e3a5-80f3-4f01-b1f8-615038840117 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:20.093318 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1192/2372] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:30:20 2022] PUT /identity/v3/projects/b8022c2f5f8845a7b96f96b9ea13ec46/users/c3c2ed10924947e6b355783ee3be122b/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:30:20.100392 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7877e4f-3398-445c-ace9-7e088424ed56 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:20.100800 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7877e4f-3398-445c-ace9-7e088424ed56 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:20.101192 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7877e4f-3398-445c-ace9-7e088424ed56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:20.150198 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-d7877e4f-3398-445c-ace9-7e088424ed56 None None] MFA Rules not processed for user `cedaf156c5584d62beedde598c94f322`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:30:20.150198 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-d7877e4f-3398-445c-ace9-7e088424ed56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:20.169200 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1181/2373] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:30:19 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:30:20.179330 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c2620-1572-431b-9310-18ecfdc93b55 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:20.180026 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c2620-1572-431b-9310-18ecfdc93b55 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:20.180502 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c2620-1572-431b-9310-18ecfdc93b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:20.228163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-1f8c2620-1572-431b-9310-18ecfdc93b55 None None] MFA Rules not processed for user `c3c2ed10924947e6b355783ee3be122b`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:30:20.230109 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1f8c2620-1572-431b-9310-18ecfdc93b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:20.277734 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1193/2374] 10.222.0.26 () {58 vars in 1009 bytes} [Fri Sep 9 20:30:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 09 20:30:20.298705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7877e4f-3398-445c-ace9-7e088424ed56 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:20.300684 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-d7877e4f-3398-445c-ace9-7e088424ed56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:20.367126 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1182/2375] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:30:20 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:30:20.383877 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c2620-1572-431b-9310-18ecfdc93b55 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:20.386454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1f8c2620-1572-431b-9310-18ecfdc93b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:20.404488 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:30:20.439420 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7877e4f-3398-445c-ace9-7e088424ed56 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:20.457590 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb01a4e5-11b0-4df3-8fb1-fee21182c17e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb62d06f7e94e9295c54504ddabb1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:20.458432 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01a4e5-11b0-4df3-8fb1-fee21182c17e service swift] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:20.459371 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01a4e5-11b0-4df3-8fb1-fee21182c17e service swift] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:20.459816 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01a4e5-11b0-4df3-8fb1-fee21182c17e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:20.473815 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c2620-1572-431b-9310-18ecfdc93b55 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:20.486256 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b8c4ad-2606-499d-bc5a-a5386d45316c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:20.487414 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8c4ad-2606-499d-bc5a-a5386d45316c service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:20.487693 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8c4ad-2606-499d-bc5a-a5386d45316c service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:20.487962 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8c4ad-2606-499d-bc5a-a5386d45316c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:20.490652 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb01a4e5-11b0-4df3-8fb1-fee21182c17e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:20.492571 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb01a4e5-11b0-4df3-8fb1-fee21182c17e service swift] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:20.511941 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1194/2376] 10.222.0.26 () {60 vars in 1384 bytes} [Fri Sep 9 20:30:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 20:30:20.572942 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b8c4ad-2606-499d-bc5a-a5386d45316c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:20.574599 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b8c4ad-2606-499d-bc5a-a5386d45316c service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:20.595980 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1183/2377] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:30:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 20:30:22.847675 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb01a4e5-11b0-4df3-8fb1-fee21182c17e service swift] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:22.849423 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-cb01a4e5-11b0-4df3-8fb1-fee21182c17e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:22.908403 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb01a4e5-11b0-4df3-8fb1-fee21182c17e service swift] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:22.916811 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ddd29-ab79-4c0d-bd59-86051f59e25f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:22.917290 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ddd29-ab79-4c0d-bd59-86051f59e25f service cinder] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:22.917541 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ddd29-ab79-4c0d-bd59-86051f59e25f service cinder] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:22.917866 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ddd29-ab79-4c0d-bd59-86051f59e25f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:22.931748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:30:23.049164 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462ddd29-ab79-4c0d-bd59-86051f59e25f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:23.050763 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462ddd29-ab79-4c0d-bd59-86051f59e25f service cinder] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:23.089671 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1195/2378] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:30:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:30:24.352200 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b8c4ad-2606-499d-bc5a-a5386d45316c service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:24.366898 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b8c4ad-2606-499d-bc5a-a5386d45316c service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:24.371583 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5163d83b-93b8-4008-ac06-6cba62b9999b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:24.372582 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163d83b-93b8-4008-ac06-6cba62b9999b service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:24.373131 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163d83b-93b8-4008-ac06-6cba62b9999b service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:24.373679 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163d83b-93b8-4008-ac06-6cba62b9999b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:24.388769 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5163d83b-93b8-4008-ac06-6cba62b9999b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:24.390853 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5163d83b-93b8-4008-ac06-6cba62b9999b service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:24.415538 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1184/2379] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:30:24 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:30:29.192903 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ddd29-ab79-4c0d-bd59-86051f59e25f service cinder] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:29.388112 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ddd29-ab79-4c0d-bd59-86051f59e25f service cinder] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:29.394879 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7588b4-caec-440f-8827-6b5ab433415e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:29.395765 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7588b4-caec-440f-8827-6b5ab433415e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:29.396197 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7588b4-caec-440f-8827-6b5ab433415e admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:29.396651 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7588b4-caec-440f-8827-6b5ab433415e admin admin] PATH_INFO: `/v3/users/662b9dab25a347108debeaf2265da8d8` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:29.398016 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7588b4-caec-440f-8827-6b5ab433415e admin admin] RBAC: Authorizing `identity:delete_user(user_id=662b9dab25a347108debeaf2265da8d8)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:29.400286 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7588b4-caec-440f-8827-6b5ab433415e admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:29.524980 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-8b7588b4-caec-440f-8827-6b5ab433415e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '662b9dab25a347108debeaf2265da8d8'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:29.553493 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-8b7588b4-caec-440f-8827-6b5ab433415e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '662b9dab25a347108debeaf2265da8d8'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:29.586235 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-8b7588b4-caec-440f-8827-6b5ab433415e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '662b9dab25a347108debeaf2265da8d8'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:29.594022 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-8b7588b4-caec-440f-8827-6b5ab433415e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '662b9dab25a347108debeaf2265da8d8'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:29.597316 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-8b7588b4-caec-440f-8827-6b5ab433415e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '662b9dab25a347108debeaf2265da8d8'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:29.613057 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1196/2380] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:30:29 2022] DELETE /identity/v3/users/662b9dab25a347108debeaf2265da8d8 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:30:29.626458 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070795dd-578f-469d-8b10-dd78a1eb0344 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:29.627151 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070795dd-578f-469d-8b10-dd78a1eb0344 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:29.628052 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070795dd-578f-469d-8b10-dd78a1eb0344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:29.780789 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-070795dd-578f-469d-8b10-dd78a1eb0344 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:30:29.782459 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-070795dd-578f-469d-8b10-dd78a1eb0344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:29.948512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1185/2381] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:30:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:30:29.955929 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7588b4-caec-440f-8827-6b5ab433415e admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:29.958063 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-8b7588b4-caec-440f-8827-6b5ab433415e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:30.067716 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:30:30.125685 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7588b4-caec-440f-8827-6b5ab433415e admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:30.130744 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6ddb72-728e-4af1-b972-4b533f55fd3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:30.132030 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ddb72-728e-4af1-b972-4b533f55fd3f service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:30.132030 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ddb72-728e-4af1-b972-4b533f55fd3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:30.132290 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ddb72-728e-4af1-b972-4b533f55fd3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:30.156187 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6ddb72-728e-4af1-b972-4b533f55fd3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:30.157835 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6ddb72-728e-4af1-b972-4b533f55fd3f service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:30.176675 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1197/2382] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:30:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:30:30.732064 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070795dd-578f-469d-8b10-dd78a1eb0344 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:30.734811 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-070795dd-578f-469d-8b10-dd78a1eb0344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:30.764716 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070795dd-578f-469d-8b10-dd78a1eb0344 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:30.778653 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be306cd-1d22-4c61-97e6-785ee43d1dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:30.779526 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be306cd-1d22-4c61-97e6-785ee43d1dc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:30.781053 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be306cd-1d22-4c61-97e6-785ee43d1dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:30.790544 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be306cd-1d22-4c61-97e6-785ee43d1dc4 admin admin] PATH_INFO: `/v3/projects/087796fe88654e079c9546b126df8f83` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:30.792225 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be306cd-1d22-4c61-97e6-785ee43d1dc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=087796fe88654e079c9546b126df8f83)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:30.794507 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be306cd-1d22-4c61-97e6-785ee43d1dc4 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:30.917028 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:30:30.926558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3be306cd-1d22-4c61-97e6-785ee43d1dc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '087796fe88654e079c9546b126df8f83', 'request_id': 'req-3be306cd-1d22-4c61-97e6-785ee43d1dc4'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:30.945965 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3be306cd-1d22-4c61-97e6-785ee43d1dc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '087796fe88654e079c9546b126df8f83', 'request_id': 'req-3be306cd-1d22-4c61-97e6-785ee43d1dc4'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:30.966869 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3be306cd-1d22-4c61-97e6-785ee43d1dc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '087796fe88654e079c9546b126df8f83', 'request_id': 'req-3be306cd-1d22-4c61-97e6-785ee43d1dc4'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:30.990083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3be306cd-1d22-4c61-97e6-785ee43d1dc4 admin admin] The token cache is being invalidate because project 087796fe88654e079c9546b126df8f83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:30:30.990460 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3be306cd-1d22-4c61-97e6-785ee43d1dc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:30.992309 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3be306cd-1d22-4c61-97e6-785ee43d1dc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:30.999581 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1186/2383] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:30:30 2022] DELETE /identity/v3/projects/087796fe88654e079c9546b126df8f83 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:30:31.006063 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6ddb72-728e-4af1-b972-4b533f55fd3f service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:31.031075 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6ddb72-728e-4af1-b972-4b533f55fd3f service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:31.302300 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf3fb0-39ec-49cf-8e1a-4606cea0e6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:31.303246 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf3fb0-39ec-49cf-8e1a-4606cea0e6b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:31.303694 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf3fb0-39ec-49cf-8e1a-4606cea0e6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:31.306170 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf3fb0-39ec-49cf-8e1a-4606cea0e6b4 admin admin] PATH_INFO: `/v3/users/612d311905d243f0bd0557ff5f65bd13` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:31.307241 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaf3fb0-39ec-49cf-8e1a-4606cea0e6b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=612d311905d243f0bd0557ff5f65bd13)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:31.330460 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaf3fb0-39ec-49cf-8e1a-4606cea0e6b4 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:31.472577 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-ceaf3fb0-39ec-49cf-8e1a-4606cea0e6b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '612d311905d243f0bd0557ff5f65bd13'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:31.512042 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-ceaf3fb0-39ec-49cf-8e1a-4606cea0e6b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '612d311905d243f0bd0557ff5f65bd13'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:31.533768 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-ceaf3fb0-39ec-49cf-8e1a-4606cea0e6b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '612d311905d243f0bd0557ff5f65bd13'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:31.553141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-ceaf3fb0-39ec-49cf-8e1a-4606cea0e6b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '612d311905d243f0bd0557ff5f65bd13'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:31.554895 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-ceaf3fb0-39ec-49cf-8e1a-4606cea0e6b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '612d311905d243f0bd0557ff5f65bd13'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:31.576767 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1198/2384] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:30:31 2022] DELETE /identity/v3/users/612d311905d243f0bd0557ff5f65bd13 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:30:32.091586 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be306cd-1d22-4c61-97e6-785ee43d1dc4 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:32.095531 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3be306cd-1d22-4c61-97e6-785ee43d1dc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:32.122975 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be306cd-1d22-4c61-97e6-785ee43d1dc4 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:32.249909 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d4c231-3f8b-410b-b6ba-040ce842f3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:32.255001 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4c231-3f8b-410b-b6ba-040ce842f3f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:32.255329 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4c231-3f8b-410b-b6ba-040ce842f3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:32.255582 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4c231-3f8b-410b-b6ba-040ce842f3f4 admin admin] PATH_INFO: `/v3/projects/592bb0bed89644c6bf68571805944a71` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:32.260808 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d4c231-3f8b-410b-b6ba-040ce842f3f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=592bb0bed89644c6bf68571805944a71)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:32.263168 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d4c231-3f8b-410b-b6ba-040ce842f3f4 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:32.341729 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-08d4c231-3f8b-410b-b6ba-040ce842f3f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '592bb0bed89644c6bf68571805944a71', 'request_id': 'req-08d4c231-3f8b-410b-b6ba-040ce842f3f4'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:32.348630 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-08d4c231-3f8b-410b-b6ba-040ce842f3f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '592bb0bed89644c6bf68571805944a71', 'request_id': 'req-08d4c231-3f8b-410b-b6ba-040ce842f3f4'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:32.357016 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-08d4c231-3f8b-410b-b6ba-040ce842f3f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '592bb0bed89644c6bf68571805944a71', 'request_id': 'req-08d4c231-3f8b-410b-b6ba-040ce842f3f4'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:32.385013 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-08d4c231-3f8b-410b-b6ba-040ce842f3f4 admin admin] The token cache is being invalidate because project 592bb0bed89644c6bf68571805944a71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:30:32.385764 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-08d4c231-3f8b-410b-b6ba-040ce842f3f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:32.386825 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-08d4c231-3f8b-410b-b6ba-040ce842f3f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:32.388749 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1187/2385] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:30:32 2022] DELETE /identity/v3/projects/592bb0bed89644c6bf68571805944a71 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:30:32.396197 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf3fb0-39ec-49cf-8e1a-4606cea0e6b4 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:32.403147 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-ceaf3fb0-39ec-49cf-8e1a-4606cea0e6b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:32.425991 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf3fb0-39ec-49cf-8e1a-4606cea0e6b4 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:32.533492 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:30:32.562453 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8aaf5f-27d1-4421-8657-d0b3513d8c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:32.564924 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8aaf5f-27d1-4421-8657-d0b3513d8c55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:32.565414 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8aaf5f-27d1-4421-8657-d0b3513d8c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:32.565845 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8aaf5f-27d1-4421-8657-d0b3513d8c55 admin admin] PATH_INFO: `/v3/users/c3c2ed10924947e6b355783ee3be122b` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:32.572574 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8aaf5f-27d1-4421-8657-d0b3513d8c55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3c2ed10924947e6b355783ee3be122b)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:32.575048 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8aaf5f-27d1-4421-8657-d0b3513d8c55 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:32.691126 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-cd8aaf5f-27d1-4421-8657-d0b3513d8c55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3c2ed10924947e6b355783ee3be122b'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:32.710080 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-cd8aaf5f-27d1-4421-8657-d0b3513d8c55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3c2ed10924947e6b355783ee3be122b'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:32.719634 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-cd8aaf5f-27d1-4421-8657-d0b3513d8c55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3c2ed10924947e6b355783ee3be122b'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:32.743651 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-cd8aaf5f-27d1-4421-8657-d0b3513d8c55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3c2ed10924947e6b355783ee3be122b'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:32.745636 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-cd8aaf5f-27d1-4421-8657-d0b3513d8c55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3c2ed10924947e6b355783ee3be122b'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:32.767931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1199/2386] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:30:32 2022] DELETE /identity/v3/users/c3c2ed10924947e6b355783ee3be122b => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:30:33.390655 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d4c231-3f8b-410b-b6ba-040ce842f3f4 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:33.395070 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-08d4c231-3f8b-410b-b6ba-040ce842f3f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:33.465302 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d4c231-3f8b-410b-b6ba-040ce842f3f4 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:33.610460 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af233f29-1082-448b-b5de-4ceecbb84de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:33.611097 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af233f29-1082-448b-b5de-4ceecbb84de9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:33.611378 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af233f29-1082-448b-b5de-4ceecbb84de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:33.611660 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af233f29-1082-448b-b5de-4ceecbb84de9 admin admin] PATH_INFO: `/v3/projects/b8022c2f5f8845a7b96f96b9ea13ec46` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:33.612953 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af233f29-1082-448b-b5de-4ceecbb84de9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8022c2f5f8845a7b96f96b9ea13ec46)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:33.617627 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af233f29-1082-448b-b5de-4ceecbb84de9 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:33.632302 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:30:33.710512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-af233f29-1082-448b-b5de-4ceecbb84de9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8022c2f5f8845a7b96f96b9ea13ec46', 'request_id': 'req-af233f29-1082-448b-b5de-4ceecbb84de9'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:33.718619 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-af233f29-1082-448b-b5de-4ceecbb84de9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8022c2f5f8845a7b96f96b9ea13ec46', 'request_id': 'req-af233f29-1082-448b-b5de-4ceecbb84de9'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:33.728385 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-af233f29-1082-448b-b5de-4ceecbb84de9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8022c2f5f8845a7b96f96b9ea13ec46', 'request_id': 'req-af233f29-1082-448b-b5de-4ceecbb84de9'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:33.759168 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-af233f29-1082-448b-b5de-4ceecbb84de9 admin admin] The token cache is being invalidate because project b8022c2f5f8845a7b96f96b9ea13ec46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:30:33.759637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-af233f29-1082-448b-b5de-4ceecbb84de9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:33.760385 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-af233f29-1082-448b-b5de-4ceecbb84de9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:33.762033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1188/2387] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:30:33 2022] DELETE /identity/v3/projects/b8022c2f5f8845a7b96f96b9ea13ec46 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:30:33.767347 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfbc4f1-9a95-441e-9259-8a26d05ef38b None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:33.767874 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfbc4f1-9a95-441e-9259-8a26d05ef38b None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:33.775841 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfbc4f1-9a95-441e-9259-8a26d05ef38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:33.818251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-3cfbc4f1-9a95-441e-9259-8a26d05ef38b None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:30:33.822799 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-3cfbc4f1-9a95-441e-9259-8a26d05ef38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:33.959850 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1200/2388] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:30:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:30:33.967761 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981fe883-f30f-4ec5-8e86-f664ce651f9b None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:33.968294 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981fe883-f30f-4ec5-8e86-f664ce651f9b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:33.968664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981fe883-f30f-4ec5-8e86-f664ce651f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:34.040111 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-981fe883-f30f-4ec5-8e86-f664ce651f9b None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:30:34.041488 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-981fe883-f30f-4ec5-8e86-f664ce651f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:34.073098 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1189/2389] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:30:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:30:34.075571 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfbc4f1-9a95-441e-9259-8a26d05ef38b None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:34.093593 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfbc4f1-9a95-441e-9259-8a26d05ef38b None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:34.119823 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dadee94-1c02-4fc1-91eb-fb7fbc0383e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:34.120641 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dadee94-1c02-4fc1-91eb-fb7fbc0383e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:34.120921 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dadee94-1c02-4fc1-91eb-fb7fbc0383e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:34.121195 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dadee94-1c02-4fc1-91eb-fb7fbc0383e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:34.122043 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dadee94-1c02-4fc1-91eb-fb7fbc0383e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:30:34.122502 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dadee94-1c02-4fc1-91eb-fb7fbc0383e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:34.124900 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dadee94-1c02-4fc1-91eb-fb7fbc0383e9 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:34.155235 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1201/2390] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:30:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:30:34.161445 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981fe883-f30f-4ec5-8e86-f664ce651f9b None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:34.172553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981fe883-f30f-4ec5-8e86-f664ce651f9b None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:34.187818 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c6481-5ad2-49af-9ef3-dde2656cb61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:34.188486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c6481-5ad2-49af-9ef3-dde2656cb61c admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:34.188764 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c6481-5ad2-49af-9ef3-dde2656cb61c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:34.189073 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c6481-5ad2-49af-9ef3-dde2656cb61c admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:34.189719 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446c6481-5ad2-49af-9ef3-dde2656cb61c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:34.192985 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446c6481-5ad2-49af-9ef3-dde2656cb61c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:34.277966 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1190/2391] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:30:34 2022] POST /identity/v3/projects => generated 369 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:30:34.278164 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dadee94-1c02-4fc1-91eb-fb7fbc0383e9 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:34.293852 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dadee94-1c02-4fc1-91eb-fb7fbc0383e9 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:34.445521 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:30:34.472243 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d9cf06-dcc3-40df-b204-cc85e90baa72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:34.473869 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d9cf06-dcc3-40df-b204-cc85e90baa72 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:34.474397 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d9cf06-dcc3-40df-b204-cc85e90baa72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:34.476020 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d9cf06-dcc3-40df-b204-cc85e90baa72 admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:34.476695 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d9cf06-dcc3-40df-b204-cc85e90baa72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:34.478730 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d9cf06-dcc3-40df-b204-cc85e90baa72 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:34.527319 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1202/2392] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:30:34 2022] POST /identity/v3/users => generated 285 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:30:34.541842 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c6481-5ad2-49af-9ef3-dde2656cb61c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:34.586924 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c6481-5ad2-49af-9ef3-dde2656cb61c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:34.614919 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec2f5c9-cea9-47cd-b3dd-0daae31094c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:34.614919 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2f5c9-cea9-47cd-b3dd-0daae31094c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:34.614919 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2f5c9-cea9-47cd-b3dd-0daae31094c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:34.614919 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2f5c9-cea9-47cd-b3dd-0daae31094c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:34.615382 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec2f5c9-cea9-47cd-b3dd-0daae31094c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:34.615382 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec2f5c9-cea9-47cd-b3dd-0daae31094c7 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:34.641315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1191/2393] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:30:34.644136 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d9cf06-dcc3-40df-b204-cc85e90baa72 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:34.706280 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d9cf06-dcc3-40df-b204-cc85e90baa72 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:34.718239 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80957edc-ffa8-4c44-a886-92012e3464df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:34.719268 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80957edc-ffa8-4c44-a886-92012e3464df admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:34.720048 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80957edc-ffa8-4c44-a886-92012e3464df admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:34.720545 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80957edc-ffa8-4c44-a886-92012e3464df admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:34.721494 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80957edc-ffa8-4c44-a886-92012e3464df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:34.723815 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80957edc-ffa8-4c44-a886-92012e3464df admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:34.749278 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1203/2394] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:30:34.754555 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec2f5c9-cea9-47cd-b3dd-0daae31094c7 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:34.783715 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec2f5c9-cea9-47cd-b3dd-0daae31094c7 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:34.794557 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf69200-56d0-4c82-ac96-c74aea3f138b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:34.795408 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf69200-56d0-4c82-ac96-c74aea3f138b admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:34.796036 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf69200-56d0-4c82-ac96-c74aea3f138b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:34.796505 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf69200-56d0-4c82-ac96-c74aea3f138b admin admin] PATH_INFO: `/v3/projects/f9e3cb1012594009b1fcc30c49425fd7/users/89bc6dcba754468bbfea2ea16695d57f/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:34.881934 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf69200-56d0-4c82-ac96-c74aea3f138b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9e3cb1012594009b1fcc30c49425fd7, user_id=89bc6dcba754468bbfea2ea16695d57f, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:34.905779 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf69200-56d0-4c82-ac96-c74aea3f138b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:34.957367 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd982fb6-ff0b-49b4-bb41-0aeaea240eff None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:34.957367 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd982fb6-ff0b-49b4-bb41-0aeaea240eff None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:34.957367 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd982fb6-ff0b-49b4-bb41-0aeaea240eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:34.967399 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1192/2395] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:30:34 2022] PUT /identity/v3/projects/f9e3cb1012594009b1fcc30c49425fd7/users/89bc6dcba754468bbfea2ea16695d57f/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:30:35.041626 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-bd982fb6-ff0b-49b4-bb41-0aeaea240eff None None] MFA Rules not processed for user `89bc6dcba754468bbfea2ea16695d57f`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:30:35.056081 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-bd982fb6-ff0b-49b4-bb41-0aeaea240eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:35.232136 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1204/2396] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:30:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:30:36.035724 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf69200-56d0-4c82-ac96-c74aea3f138b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:36.037606 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7bf69200-56d0-4c82-ac96-c74aea3f138b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:36.142894 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9cbf6-f69b-41a6-acfa-f0b3e9fe01f0 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:36.143449 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9cbf6-f69b-41a6-acfa-f0b3e9fe01f0 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:36.143449 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9cbf6-f69b-41a6-acfa-f0b3e9fe01f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:36.152357 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf69200-56d0-4c82-ac96-c74aea3f138b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:36.157214 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e440960d-42a3-4e5b-831a-81d4a5befbeb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5298486310004b63b51116dbc747e31d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:36.158180 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440960d-42a3-4e5b-831a-81d4a5befbeb service glance] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:36.158662 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440960d-42a3-4e5b-831a-81d4a5befbeb service glance] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:36.159162 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440960d-42a3-4e5b-831a-81d4a5befbeb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:36.172479 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e440960d-42a3-4e5b-831a-81d4a5befbeb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:36.175397 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e440960d-42a3-4e5b-831a-81d4a5befbeb service glance] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:36.189183 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1193/2397] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:30:36 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:30:36.197066 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-09e9cbf6-f69b-41a6-acfa-f0b3e9fe01f0 None None] MFA Rules not processed for user `ec90aaf1a60746f69e5478987ab86bc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:30:36.200660 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-09e9cbf6-f69b-41a6-acfa-f0b3e9fe01f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:36.342959 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1205/2398] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:30:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:30:36.706590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1d3c5-a2f5-4637-8350-28d1d21e004a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:36.706929 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1d3c5-a2f5-4637-8350-28d1d21e004a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:36.707208 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1d3c5-a2f5-4637-8350-28d1d21e004a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:36.760701 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-77a1d3c5-a2f5-4637-8350-28d1d21e004a None None] MFA Rules not processed for user `cedaf156c5584d62beedde598c94f322`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:30:36.763696 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-77a1d3c5-a2f5-4637-8350-28d1d21e004a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:36.832466 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:30:36.878296 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1194/2399] 10.222.0.26 () {58 vars in 1009 bytes} [Fri Sep 9 20:30:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:30:36.893149 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e9cbf6-f69b-41a6-acfa-f0b3e9fe01f0 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:36.895010 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-09e9cbf6-f69b-41a6-acfa-f0b3e9fe01f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:37.033326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e9cbf6-f69b-41a6-acfa-f0b3e9fe01f0 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:37.039211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec1356-f3bf-4a74-8c17-ea6c222a89ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb62d06f7e94e9295c54504ddabb1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:37.040437 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec1356-f3bf-4a74-8c17-ea6c222a89ef service swift] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:37.040777 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec1356-f3bf-4a74-8c17-ea6c222a89ef service swift] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:37.041107 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec1356-f3bf-4a74-8c17-ea6c222a89ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:37.050876 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec1356-f3bf-4a74-8c17-ea6c222a89ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:37.052488 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec1356-f3bf-4a74-8c17-ea6c222a89ef service swift] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:37.062383 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:30:37.062663 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1206/2400] 10.222.0.26 () {60 vars in 1384 bytes} [Fri Sep 9 20:30:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 20:30:38.652759 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbdaff2-c174-454f-9784-1de38a0907b0 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:38.653232 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbdaff2-c174-454f-9784-1de38a0907b0 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:38.653367 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbdaff2-c174-454f-9784-1de38a0907b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:38.712600 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-dfbdaff2-c174-454f-9784-1de38a0907b0 None None] MFA Rules not processed for user `cedaf156c5584d62beedde598c94f322`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:30:38.714120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-dfbdaff2-c174-454f-9784-1de38a0907b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:38.737468 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1195/2401] 10.222.0.26 () {58 vars in 967 bytes} [Fri Sep 9 20:30:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:30:38.755891 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec1356-f3bf-4a74-8c17-ea6c222a89ef service swift] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:38.775868 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec1356-f3bf-4a74-8c17-ea6c222a89ef service swift] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:38.787725 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5acf6ab-3d64-4a3e-8749-e8551a6fec53 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb62d06f7e94e9295c54504ddabb1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:38.788777 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5acf6ab-3d64-4a3e-8749-e8551a6fec53 service swift] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:38.789887 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5acf6ab-3d64-4a3e-8749-e8551a6fec53 service swift] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:38.790506 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5acf6ab-3d64-4a3e-8749-e8551a6fec53 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:38.812932 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5acf6ab-3d64-4a3e-8749-e8551a6fec53 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:38.820207 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5acf6ab-3d64-4a3e-8749-e8551a6fec53 service swift] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:38.835720 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1207/2402] 10.222.0.26 () {60 vars in 1384 bytes} [Fri Sep 9 20:30:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 20:30:42.649766 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e4bda-960b-43f7-8af0-4dcc79b17052 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:42.650757 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e4bda-960b-43f7-8af0-4dcc79b17052 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:42.651185 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e4bda-960b-43f7-8af0-4dcc79b17052 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:42.659486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1196/2403] 10.222.0.26 () {54 vars in 921 bytes} [Fri Sep 9 20:30:42 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:30:42.681528 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ed034-6503-40da-831d-e52707460d02 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:42.682387 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ed034-6503-40da-831d-e52707460d02 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:42.682935 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ed034-6503-40da-831d-e52707460d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:42.727868 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-322ed034-6503-40da-831d-e52707460d02 None None] MFA Rules not processed for user `26d24cf0a21d491082b0c2f11e749446`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:30:42.742081 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-322ed034-6503-40da-831d-e52707460d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:42.856251 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e4bda-960b-43f7-8af0-4dcc79b17052 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:42.858219 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-652e4bda-960b-43f7-8af0-4dcc79b17052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:42.936770 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1208/2404] 10.222.0.26 () {58 vars in 1018 bytes} [Fri Sep 9 20:30:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:30:42.950806 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322ed034-6503-40da-831d-e52707460d02 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:42.953935 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-322ed034-6503-40da-831d-e52707460d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:42.973724 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322ed034-6503-40da-831d-e52707460d02 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:42.987628 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870680ab-b61b-4611-8a96-a99257009df4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26d24cf0a21d491082b0c2f11e749446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:42.988668 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870680ab-b61b-4611-8a96-a99257009df4 service placement] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:42.990453 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870680ab-b61b-4611-8a96-a99257009df4 service placement] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:42.991181 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870680ab-b61b-4611-8a96-a99257009df4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:43.001887 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870680ab-b61b-4611-8a96-a99257009df4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:43.003366 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e4bda-960b-43f7-8af0-4dcc79b17052 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:43.004555 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870680ab-b61b-4611-8a96-a99257009df4 service placement] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:43.013373 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a69fe-2d7a-4c3c-a117-79bc81f4ba60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:43.014331 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a69fe-2d7a-4c3c-a117-79bc81f4ba60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:43.014837 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a69fe-2d7a-4c3c-a117-79bc81f4ba60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:43.015767 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a69fe-2d7a-4c3c-a117-79bc81f4ba60 admin admin] PATH_INFO: `/v3/users/892bbd02129b417c987f4d429e21a833` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:43.017686 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029a69fe-2d7a-4c3c-a117-79bc81f4ba60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=892bbd02129b417c987f4d429e21a833)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:43.021924 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029a69fe-2d7a-4c3c-a117-79bc81f4ba60 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:43.026074 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1209/2405] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:30:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:30:43.131110 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-029a69fe-2d7a-4c3c-a117-79bc81f4ba60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '892bbd02129b417c987f4d429e21a833'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:43.142905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-029a69fe-2d7a-4c3c-a117-79bc81f4ba60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '892bbd02129b417c987f4d429e21a833'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:43.169120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-029a69fe-2d7a-4c3c-a117-79bc81f4ba60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '892bbd02129b417c987f4d429e21a833'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:43.190374 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-029a69fe-2d7a-4c3c-a117-79bc81f4ba60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '892bbd02129b417c987f4d429e21a833'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:43.192348 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-029a69fe-2d7a-4c3c-a117-79bc81f4ba60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '892bbd02129b417c987f4d429e21a833'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:43.208686 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1197/2406] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:30:42 2022] DELETE /identity/v3/users/892bbd02129b417c987f4d429e21a833 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:30:43.612336 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870680ab-b61b-4611-8a96-a99257009df4 service placement] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:43.643747 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870680ab-b61b-4611-8a96-a99257009df4 service placement] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:43.837617 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec685fb-30e1-453f-8197-6429c17d27e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:43.838754 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec685fb-30e1-453f-8197-6429c17d27e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:43.840014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec685fb-30e1-453f-8197-6429c17d27e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:43.840120 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec685fb-30e1-453f-8197-6429c17d27e3 admin admin] PATH_INFO: `/v3/projects/32c862d7b7b242f699139ac0821867a8` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:43.844694 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec685fb-30e1-453f-8197-6429c17d27e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32c862d7b7b242f699139ac0821867a8)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:43.849336 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec685fb-30e1-453f-8197-6429c17d27e3 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:43.990618 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-bec685fb-30e1-453f-8197-6429c17d27e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32c862d7b7b242f699139ac0821867a8', 'request_id': 'req-bec685fb-30e1-453f-8197-6429c17d27e3'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:44.011527 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-bec685fb-30e1-453f-8197-6429c17d27e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32c862d7b7b242f699139ac0821867a8', 'request_id': 'req-bec685fb-30e1-453f-8197-6429c17d27e3'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:44.020000 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-bec685fb-30e1-453f-8197-6429c17d27e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32c862d7b7b242f699139ac0821867a8', 'request_id': 'req-bec685fb-30e1-453f-8197-6429c17d27e3'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:44.047849 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:30:44.057867 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-bec685fb-30e1-453f-8197-6429c17d27e3 admin admin] The token cache is being invalidate because project 32c862d7b7b242f699139ac0821867a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:30:44.058613 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-bec685fb-30e1-453f-8197-6429c17d27e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:44.063675 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-bec685fb-30e1-453f-8197-6429c17d27e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:44.069343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1210/2407] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:30:43 2022] DELETE /identity/v3/projects/32c862d7b7b242f699139ac0821867a8 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:30:44.077372 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0f713-6096-4197-a630-cb03401a98e7 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:44.077924 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0f713-6096-4197-a630-cb03401a98e7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:44.077924 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0f713-6096-4197-a630-cb03401a98e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:44.142393 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-88a0f713-6096-4197-a630-cb03401a98e7 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:30:44.144117 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-88a0f713-6096-4197-a630-cb03401a98e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:44.247314 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:30:44.275468 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40b92f-29b9-441c-b56b-d6e636c13988 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:44.277762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40b92f-29b9-441c-b56b-d6e636c13988 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:44.278328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40b92f-29b9-441c-b56b-d6e636c13988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:44.287088 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1198/2408] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:30:44.314312 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-ba40b92f-29b9-441c-b56b-d6e636c13988 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:30:44.316008 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-ba40b92f-29b9-441c-b56b-d6e636c13988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:44.333208 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a0f713-6096-4197-a630-cb03401a98e7 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:44.346282 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1211/2409] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:30:44.351490 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a0f713-6096-4197-a630-cb03401a98e7 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:44.357387 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed0c12-3f1f-48cd-866c-d42f45e07e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:44.358005 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed0c12-3f1f-48cd-866c-d42f45e07e98 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:44.358280 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed0c12-3f1f-48cd-866c-d42f45e07e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:44.358549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed0c12-3f1f-48cd-866c-d42f45e07e98 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:44.359200 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aed0c12-3f1f-48cd-866c-d42f45e07e98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:30:44.359496 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aed0c12-3f1f-48cd-866c-d42f45e07e98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:44.361844 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aed0c12-3f1f-48cd-866c-d42f45e07e98 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:44.389046 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba40b92f-29b9-441c-b56b-d6e636c13988 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:44.397288 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1199/2410] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:30:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:30:44.410806 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba40b92f-29b9-441c-b56b-d6e636c13988 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:44.417070 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b3400-9fed-4893-95b8-25a5c81dada8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:44.418049 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b3400-9fed-4893-95b8-25a5c81dada8 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:44.418698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b3400-9fed-4893-95b8-25a5c81dada8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:44.419320 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b3400-9fed-4893-95b8-25a5c81dada8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:44.420569 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581b3400-9fed-4893-95b8-25a5c81dada8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:44.422890 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581b3400-9fed-4893-95b8-25a5c81dada8 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:44.467508 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1212/2411] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:30:44 2022] POST /identity/v3/projects => generated 395 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:30:44.468956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed0c12-3f1f-48cd-866c-d42f45e07e98 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:44.480650 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed0c12-3f1f-48cd-866c-d42f45e07e98 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:44.569323 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d83fc9-cb9f-472b-8bad-4af36e3ba3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:44.569905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d83fc9-cb9f-472b-8bad-4af36e3ba3a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:44.570968 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d83fc9-cb9f-472b-8bad-4af36e3ba3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:44.571316 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d83fc9-cb9f-472b-8bad-4af36e3ba3a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:44.572068 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d83fc9-cb9f-472b-8bad-4af36e3ba3a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:44.574299 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d83fc9-cb9f-472b-8bad-4af36e3ba3a2 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:44.617148 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1200/2412] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:30:44 2022] POST /identity/v3/users => generated 298 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:30:44.621357 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b3400-9fed-4893-95b8-25a5c81dada8 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:44.632437 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b3400-9fed-4893-95b8-25a5c81dada8 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:44.642120 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dabad9e-eecd-4126-a416-1391faddd6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:44.642831 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabad9e-eecd-4126-a416-1391faddd6a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:44.643169 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabad9e-eecd-4126-a416-1391faddd6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:44.643502 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabad9e-eecd-4126-a416-1391faddd6a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:44.644373 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dabad9e-eecd-4126-a416-1391faddd6a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:44.647794 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dabad9e-eecd-4126-a416-1391faddd6a3 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:44.666835 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1213/2413] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:30:44.682987 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d83fc9-cb9f-472b-8bad-4af36e3ba3a2 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:44.703477 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d83fc9-cb9f-472b-8bad-4af36e3ba3a2 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:44.715589 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf434b7-c7d0-4dc4-9fd6-a820be809f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:44.716222 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf434b7-c7d0-4dc4-9fd6-a820be809f2a admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:44.716508 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf434b7-c7d0-4dc4-9fd6-a820be809f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:44.716785 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf434b7-c7d0-4dc4-9fd6-a820be809f2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:44.717468 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf434b7-c7d0-4dc4-9fd6-a820be809f2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:44.719067 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf434b7-c7d0-4dc4-9fd6-a820be809f2a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:44.743800 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1201/2414] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:30:44.744934 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dabad9e-eecd-4126-a416-1391faddd6a3 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:44.757959 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dabad9e-eecd-4126-a416-1391faddd6a3 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:44.768469 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6587073c-073f-43e7-a98f-9eea037076d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:44.769146 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587073c-073f-43e7-a98f-9eea037076d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:44.769596 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587073c-073f-43e7-a98f-9eea037076d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:44.769892 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587073c-073f-43e7-a98f-9eea037076d0 admin admin] PATH_INFO: `/v3/projects/ef1b096a5c1e4cf5b5791eb00533d106/users/54d683d03a394b5e8b7e94eda0e23baf/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:44.794570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6587073c-073f-43e7-a98f-9eea037076d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef1b096a5c1e4cf5b5791eb00533d106, user_id=54d683d03a394b5e8b7e94eda0e23baf, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:44.797792 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6587073c-073f-43e7-a98f-9eea037076d0 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:44.831864 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1214/2415] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:30:44 2022] PUT /identity/v3/projects/ef1b096a5c1e4cf5b5791eb00533d106/users/54d683d03a394b5e8b7e94eda0e23baf/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:30:44.843111 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649e75a-7dbc-4940-b498-193d20762ac2 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:44.843426 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649e75a-7dbc-4940-b498-193d20762ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:44.843952 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649e75a-7dbc-4940-b498-193d20762ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:44.922192 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e649e75a-7dbc-4940-b498-193d20762ac2 None None] MFA Rules not processed for user `54d683d03a394b5e8b7e94eda0e23baf`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:30:44.923598 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e649e75a-7dbc-4940-b498-193d20762ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:45.030775 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1202/2416] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:30:44 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:30:45.045448 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6587073c-073f-43e7-a98f-9eea037076d0 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:45.048000 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6587073c-073f-43e7-a98f-9eea037076d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:45.123326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6587073c-073f-43e7-a98f-9eea037076d0 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:45.138777 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7737dc-f916-41dc-907b-77cf2b4b201e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:45.139421 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7737dc-f916-41dc-907b-77cf2b4b201e service cinder] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:45.139774 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7737dc-f916-41dc-907b-77cf2b4b201e service cinder] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:45.140049 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7737dc-f916-41dc-907b-77cf2b4b201e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:45.152983 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7737dc-f916-41dc-907b-77cf2b4b201e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:45.155475 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7737dc-f916-41dc-907b-77cf2b4b201e service cinder] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:45.165287 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1215/2417] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:30:45 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:30:50.673606 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb343973-363c-4bd3-868e-2317611de473 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:50.674509 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb343973-363c-4bd3-868e-2317611de473 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:50.674509 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb343973-363c-4bd3-868e-2317611de473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:50.705656 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-fb343973-363c-4bd3-868e-2317611de473 None None] MFA Rules not processed for user `cedaf156c5584d62beedde598c94f322`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:30:50.706910 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-fb343973-363c-4bd3-868e-2317611de473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:50.784850 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:30:50.827593 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1203/2418] 10.222.0.26 () {58 vars in 1009 bytes} [Fri Sep 9 20:30:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:30:50.866388 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7737dc-f916-41dc-907b-77cf2b4b201e service cinder] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:50.869497 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-0b7737dc-f916-41dc-907b-77cf2b4b201e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:50.979081 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7737dc-f916-41dc-907b-77cf2b4b201e service cinder] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:50.984699 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7775f30-2b46-45e1-93d0-65aa73f14631 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb62d06f7e94e9295c54504ddabb1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:50.988241 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7775f30-2b46-45e1-93d0-65aa73f14631 service swift] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:50.988241 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7775f30-2b46-45e1-93d0-65aa73f14631 service swift] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:50.988241 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7775f30-2b46-45e1-93d0-65aa73f14631 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:50.999157 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7775f30-2b46-45e1-93d0-65aa73f14631 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:51.001679 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7775f30-2b46-45e1-93d0-65aa73f14631 service swift] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:51.014235 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1216/2419] 10.222.0.26 () {60 vars in 1384 bytes} [Fri Sep 9 20:30:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 20:30:53.804573 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb343973-363c-4bd3-868e-2317611de473 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:53.870091 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb343973-363c-4bd3-868e-2317611de473 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:53.874576 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6177e4-63f8-44d1-b640-969910611788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:53.875389 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6177e4-63f8-44d1-b640-969910611788 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:53.875756 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6177e4-63f8-44d1-b640-969910611788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:53.876088 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6177e4-63f8-44d1-b640-969910611788 admin admin] PATH_INFO: `/v3/users/54d683d03a394b5e8b7e94eda0e23baf` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:53.877289 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6177e4-63f8-44d1-b640-969910611788 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54d683d03a394b5e8b7e94eda0e23baf)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:53.879253 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6177e4-63f8-44d1-b640-969910611788 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:53.931187 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5d6177e4-63f8-44d1-b640-969910611788 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54d683d03a394b5e8b7e94eda0e23baf'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:53.938228 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5d6177e4-63f8-44d1-b640-969910611788 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54d683d03a394b5e8b7e94eda0e23baf'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:53.955634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5d6177e4-63f8-44d1-b640-969910611788 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54d683d03a394b5e8b7e94eda0e23baf'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:53.965167 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5d6177e4-63f8-44d1-b640-969910611788 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54d683d03a394b5e8b7e94eda0e23baf'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:53.966394 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5d6177e4-63f8-44d1-b640-969910611788 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54d683d03a394b5e8b7e94eda0e23baf'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:53.973725 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1204/2420] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:30:53 2022] DELETE /identity/v3/users/54d683d03a394b5e8b7e94eda0e23baf => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:30:53.989315 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c242a5-06c8-4df0-9896-c9531b50e96f None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:53.989315 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c242a5-06c8-4df0-9896-c9531b50e96f None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:53.989315 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c242a5-06c8-4df0-9896-c9531b50e96f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:54.008255 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-10c242a5-06c8-4df0-9896-c9531b50e96f None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:30:54.009864 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-10c242a5-06c8-4df0-9896-c9531b50e96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:54.032703 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:30:54.077625 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1217/2421] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:30:54.088737 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6177e4-63f8-44d1-b640-969910611788 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:54.090431 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5d6177e4-63f8-44d1-b640-969910611788 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:54.153937 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6177e4-63f8-44d1-b640-969910611788 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:54.157713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0627a8-f9b8-48f2-9584-a7fde46cc939 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:54.158400 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0627a8-f9b8-48f2-9584-a7fde46cc939 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:54.158747 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0627a8-f9b8-48f2-9584-a7fde46cc939 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:54.159085 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0627a8-f9b8-48f2-9584-a7fde46cc939 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:54.171378 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0627a8-f9b8-48f2-9584-a7fde46cc939 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:54.172820 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0627a8-f9b8-48f2-9584-a7fde46cc939 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:54.183630 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1205/2422] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:30:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:30:54.484699 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c242a5-06c8-4df0-9896-c9531b50e96f None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:30:54.486295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-10c242a5-06c8-4df0-9896-c9531b50e96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:30:54.500460 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c242a5-06c8-4df0-9896-c9531b50e96f None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:30:54.508267 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6661913e-bfb2-445a-82bf-399da43741ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:30:54.508932 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661913e-bfb2-445a-82bf-399da43741ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:30:54.509753 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661913e-bfb2-445a-82bf-399da43741ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:30:54.510087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661913e-bfb2-445a-82bf-399da43741ee admin admin] PATH_INFO: `/v3/projects/ef1b096a5c1e4cf5b5791eb00533d106` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:30:54.511262 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6661913e-bfb2-445a-82bf-399da43741ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef1b096a5c1e4cf5b5791eb00533d106)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:30:54.513180 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6661913e-bfb2-445a-82bf-399da43741ee admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:30:54.572343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6661913e-bfb2-445a-82bf-399da43741ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef1b096a5c1e4cf5b5791eb00533d106', 'request_id': 'req-6661913e-bfb2-445a-82bf-399da43741ee'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:54.586380 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6661913e-bfb2-445a-82bf-399da43741ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef1b096a5c1e4cf5b5791eb00533d106', 'request_id': 'req-6661913e-bfb2-445a-82bf-399da43741ee'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:54.593359 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6661913e-bfb2-445a-82bf-399da43741ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef1b096a5c1e4cf5b5791eb00533d106', 'request_id': 'req-6661913e-bfb2-445a-82bf-399da43741ee'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:54.605574 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6661913e-bfb2-445a-82bf-399da43741ee admin admin] The token cache is being invalidate because project ef1b096a5c1e4cf5b5791eb00533d106 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:30:54.605858 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6661913e-bfb2-445a-82bf-399da43741ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:54.606991 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6661913e-bfb2-445a-82bf-399da43741ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:30:54.615802 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1218/2423] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:30:54 2022] DELETE /identity/v3/projects/ef1b096a5c1e4cf5b5791eb00533d106 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:31:04.178649 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=82,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:31:04.602258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:31:07.472366 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0627a8-f9b8-48f2-9584-a7fde46cc939 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:31:07.473910 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2e0627a8-f9b8-48f2-9584-a7fde46cc939 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:31:07.555885 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0627a8-f9b8-48f2-9584-a7fde46cc939 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:31:07.620183 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e7b32d-f5ba-427a-9bd8-d49fbcf19daf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26d24cf0a21d491082b0c2f11e749446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:31:07.620714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7b32d-f5ba-427a-9bd8-d49fbcf19daf service placement] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:07.620999 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7b32d-f5ba-427a-9bd8-d49fbcf19daf service placement] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:07.621293 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7b32d-f5ba-427a-9bd8-d49fbcf19daf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:07.623009 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-06e7b32d-f5ba-427a-9bd8-d49fbcf19daf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:31:07.634586 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e7b32d-f5ba-427a-9bd8-d49fbcf19daf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:31:07.636241 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e7b32d-f5ba-427a-9bd8-d49fbcf19daf service placement] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:31:07.646504 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1206/2424] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:31:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:31:08.623146 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570affe-e6f6-4de5-b67c-d535f0242ef3 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:08.624086 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570affe-e6f6-4de5-b67c-d535f0242ef3 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:08.624637 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570affe-e6f6-4de5-b67c-d535f0242ef3 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:08.626200 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1219/2425] 10.222.0.26 () {54 vars in 921 bytes} [Fri Sep 9 20:31:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:31:08.635637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c5f08-01af-4ae3-bb69-3b796d19af85 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:08.636168 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c5f08-01af-4ae3-bb69-3b796d19af85 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:08.636485 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c5f08-01af-4ae3-bb69-3b796d19af85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:08.659929 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f83c5f08-01af-4ae3-bb69-3b796d19af85 None None] MFA Rules not processed for user `87ece6128f9245f7b49f3e7a9faaed22`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:31:08.661060 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f83c5f08-01af-4ae3-bb69-3b796d19af85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:31:08.736789 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1207/2426] 10.222.0.26 () {58 vars in 1018 bytes} [Fri Sep 9 20:31:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:31:08.749653 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570affe-e6f6-4de5-b67c-d535f0242ef3 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:31:08.751992 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7570affe-e6f6-4de5-b67c-d535f0242ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:31:08.816999 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570affe-e6f6-4de5-b67c-d535f0242ef3 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:31:08.821056 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c40054-d2c3-49ad-a3f5-4e96257ebfbd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5298486310004b63b51116dbc747e31d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:31:08.821702 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c40054-d2c3-49ad-a3f5-4e96257ebfbd service glance] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:08.821981 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c40054-d2c3-49ad-a3f5-4e96257ebfbd service glance] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:08.822254 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c40054-d2c3-49ad-a3f5-4e96257ebfbd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:08.833088 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c40054-d2c3-49ad-a3f5-4e96257ebfbd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:31:08.834543 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c40054-d2c3-49ad-a3f5-4e96257ebfbd service glance] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:31:08.846301 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1220/2427] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:31:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:31:08.916223 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d54d6-3f74-4e74-8c14-c888fc6c938d None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:08.916432 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d54d6-3f74-4e74-8c14-c888fc6c938d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:08.916702 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d54d6-3f74-4e74-8c14-c888fc6c938d None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:08.918223 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1208/2428] 10.222.0.26 () {54 vars in 921 bytes} [Fri Sep 9 20:31:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:31:08.924024 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812548f-4c7f-497b-a65d-aa4dde21a298 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:08.924308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812548f-4c7f-497b-a65d-aa4dde21a298 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:08.924583 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812548f-4c7f-497b-a65d-aa4dde21a298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:08.946335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-2812548f-4c7f-497b-a65d-aa4dde21a298 None None] MFA Rules not processed for user `ee5c6e51f2694bf5ae7ea097cf33fb3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:31:08.947555 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-2812548f-4c7f-497b-a65d-aa4dde21a298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:31:08.999323 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1221/2429] 10.222.0.26 () {58 vars in 1018 bytes} [Fri Sep 9 20:31:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:31:09.011865 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d54d6-3f74-4e74-8c14-c888fc6c938d None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:31:09.013628 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9a9d54d6-3f74-4e74-8c14-c888fc6c938d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:31:09.027411 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d54d6-3f74-4e74-8c14-c888fc6c938d None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:31:09.031666 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9db4fe-1835-4292-8b1f-c48d213912de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:31:09.032467 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9db4fe-1835-4292-8b1f-c48d213912de service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:09.032821 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9db4fe-1835-4292-8b1f-c48d213912de service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:09.033164 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9db4fe-1835-4292-8b1f-c48d213912de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:09.052032 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9db4fe-1835-4292-8b1f-c48d213912de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:31:09.054272 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9db4fe-1835-4292-8b1f-c48d213912de service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:31:09.078838 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1209/2430] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:31:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:31:09.186002 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db7543-9286-4cd7-aac1-2ae43677a5ee None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:09.186300 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db7543-9286-4cd7-aac1-2ae43677a5ee None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:09.186577 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db7543-9286-4cd7-aac1-2ae43677a5ee None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:09.187924 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1222/2431] 10.222.0.26 () {54 vars in 921 bytes} [Fri Sep 9 20:31:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:31:09.193641 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a1eac-7607-4596-a0d5-6dd5bc6560e5 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:09.193931 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a1eac-7607-4596-a0d5-6dd5bc6560e5 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:09.194269 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a1eac-7607-4596-a0d5-6dd5bc6560e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:09.214276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f15a1eac-7607-4596-a0d5-6dd5bc6560e5 None None] MFA Rules not processed for user `87ece6128f9245f7b49f3e7a9faaed22`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:31:09.215444 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f15a1eac-7607-4596-a0d5-6dd5bc6560e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:31:09.224507 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1210/2432] 10.222.0.26 () {58 vars in 1018 bytes} [Fri Sep 9 20:31:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:31:09.235919 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db7543-9286-4cd7-aac1-2ae43677a5ee None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:31:09.237604 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-c2db7543-9286-4cd7-aac1-2ae43677a5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:31:09.252094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db7543-9286-4cd7-aac1-2ae43677a5ee None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:31:09.255623 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f851cb-e6e4-4d3a-840d-58bffe90dd09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:31:09.256210 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f851cb-e6e4-4d3a-840d-58bffe90dd09 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:09.256484 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f851cb-e6e4-4d3a-840d-58bffe90dd09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:09.256753 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f851cb-e6e4-4d3a-840d-58bffe90dd09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:09.266840 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f851cb-e6e4-4d3a-840d-58bffe90dd09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:31:09.268307 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f851cb-e6e4-4d3a-840d-58bffe90dd09 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:31:09.278722 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1223/2433] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:31:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:31:12.845840 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc23638-f179-4d21-bbd4-176a72a47ab5 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:12.846320 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc23638-f179-4d21-bbd4-176a72a47ab5 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:12.846590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc23638-f179-4d21-bbd4-176a72a47ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:12.860031 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:31:12.878238 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-fbc23638-f179-4d21-bbd4-176a72a47ab5 None None] MFA Rules not processed for user `cedaf156c5584d62beedde598c94f322`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:31:12.879776 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-fbc23638-f179-4d21-bbd4-176a72a47ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:31:12.963704 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1211/2434] 10.222.0.26 () {58 vars in 1009 bytes} [Fri Sep 9 20:31:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:31:12.989657 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f851cb-e6e4-4d3a-840d-58bffe90dd09 service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:31:12.991130 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-b2f851cb-e6e4-4d3a-840d-58bffe90dd09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:31:13.077571 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f851cb-e6e4-4d3a-840d-58bffe90dd09 service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:31:13.083492 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7892b0-a8d2-4930-8ced-5075c1bc41fe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb62d06f7e94e9295c54504ddabb1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:31:13.084109 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7892b0-a8d2-4930-8ced-5075c1bc41fe service swift] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:13.084449 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7892b0-a8d2-4930-8ced-5075c1bc41fe service swift] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:13.085323 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7892b0-a8d2-4930-8ced-5075c1bc41fe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:13.095481 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7892b0-a8d2-4930-8ced-5075c1bc41fe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:31:13.097090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7892b0-a8d2-4930-8ced-5075c1bc41fe service swift] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:31:13.105839 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1224/2435] 10.222.0.26 () {60 vars in 1384 bytes} [Fri Sep 9 20:31:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 20:31:13.117953 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc23638-f179-4d21-bbd4-176a72a47ab5 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:31:13.119812 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-fbc23638-f179-4d21-bbd4-176a72a47ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:31:13.207411 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc23638-f179-4d21-bbd4-176a72a47ab5 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:31:13.215832 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1026b-f41d-4e78-b4be-2c5e1f87b250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:31:13.216748 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1026b-f41d-4e78-b4be-2c5e1f87b250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:13.217615 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1026b-f41d-4e78-b4be-2c5e1f87b250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:13.217962 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1026b-f41d-4e78-b4be-2c5e1f87b250 admin admin] PATH_INFO: `/v3/users/89bc6dcba754468bbfea2ea16695d57f` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:13.219816 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae1026b-f41d-4e78-b4be-2c5e1f87b250 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89bc6dcba754468bbfea2ea16695d57f)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:31:13.222797 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae1026b-f41d-4e78-b4be-2c5e1f87b250 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:31:13.401487 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0ae1026b-f41d-4e78-b4be-2c5e1f87b250 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89bc6dcba754468bbfea2ea16695d57f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:31:13.436790 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0ae1026b-f41d-4e78-b4be-2c5e1f87b250 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89bc6dcba754468bbfea2ea16695d57f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:31:13.484728 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0ae1026b-f41d-4e78-b4be-2c5e1f87b250 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89bc6dcba754468bbfea2ea16695d57f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:31:13.500752 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0ae1026b-f41d-4e78-b4be-2c5e1f87b250 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89bc6dcba754468bbfea2ea16695d57f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:31:13.501918 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0ae1026b-f41d-4e78-b4be-2c5e1f87b250 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89bc6dcba754468bbfea2ea16695d57f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:31:13.518922 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1212/2436] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:31:13 2022] DELETE /identity/v3/users/89bc6dcba754468bbfea2ea16695d57f => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:31:13.525341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d19c0-8a15-41e3-a3fc-f5121df3493d None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:13.527565 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d19c0-8a15-41e3-a3fc-f5121df3493d None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:13.528334 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d19c0-8a15-41e3-a3fc-f5121df3493d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:13.570653 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-538d19c0-8a15-41e3-a3fc-f5121df3493d None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:31:13.572390 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-538d19c0-8a15-41e3-a3fc-f5121df3493d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:31:13.616664 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:31:13.696941 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1225/2437] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:31:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:31:13.706875 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1026b-f41d-4e78-b4be-2c5e1f87b250 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:31:13.708548 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0ae1026b-f41d-4e78-b4be-2c5e1f87b250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:31:13.746070 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:31:13.786357 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1026b-f41d-4e78-b4be-2c5e1f87b250 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:31:13.790667 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6580cee2-7315-4af9-a92f-d4b9d9b60839 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:31:13.791249 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580cee2-7315-4af9-a92f-d4b9d9b60839 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:13.791562 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580cee2-7315-4af9-a92f-d4b9d9b60839 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:13.791947 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580cee2-7315-4af9-a92f-d4b9d9b60839 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:13.802763 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6580cee2-7315-4af9-a92f-d4b9d9b60839 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:31:13.804265 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6580cee2-7315-4af9-a92f-d4b9d9b60839 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:31:13.815593 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1213/2438] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:31:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:31:14.258909 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538d19c0-8a15-41e3-a3fc-f5121df3493d None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:31:14.261129 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-538d19c0-8a15-41e3-a3fc-f5121df3493d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:31:14.282173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538d19c0-8a15-41e3-a3fc-f5121df3493d None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:31:14.291151 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4660d0be-fd10-4ea0-9cc8-243307dbd9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:31:14.292182 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660d0be-fd10-4ea0-9cc8-243307dbd9b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:14.292699 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660d0be-fd10-4ea0-9cc8-243307dbd9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:14.293202 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660d0be-fd10-4ea0-9cc8-243307dbd9b5 admin admin] PATH_INFO: `/v3/projects/f9e3cb1012594009b1fcc30c49425fd7` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:14.301477 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4660d0be-fd10-4ea0-9cc8-243307dbd9b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9e3cb1012594009b1fcc30c49425fd7)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:31:14.306304 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4660d0be-fd10-4ea0-9cc8-243307dbd9b5 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:31:14.409985 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-4660d0be-fd10-4ea0-9cc8-243307dbd9b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9e3cb1012594009b1fcc30c49425fd7', 'request_id': 'req-4660d0be-fd10-4ea0-9cc8-243307dbd9b5'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:31:14.425227 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-4660d0be-fd10-4ea0-9cc8-243307dbd9b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9e3cb1012594009b1fcc30c49425fd7', 'request_id': 'req-4660d0be-fd10-4ea0-9cc8-243307dbd9b5'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:31:14.439947 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-4660d0be-fd10-4ea0-9cc8-243307dbd9b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9e3cb1012594009b1fcc30c49425fd7', 'request_id': 'req-4660d0be-fd10-4ea0-9cc8-243307dbd9b5'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:31:14.455250 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-4660d0be-fd10-4ea0-9cc8-243307dbd9b5 admin admin] The token cache is being invalidate because project f9e3cb1012594009b1fcc30c49425fd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:31:14.458711 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-4660d0be-fd10-4ea0-9cc8-243307dbd9b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:31:14.458711 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-4660d0be-fd10-4ea0-9cc8-243307dbd9b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:31:14.458977 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1226/2439] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:31:14 2022] DELETE /identity/v3/projects/f9e3cb1012594009b1fcc30c49425fd7 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:31:14.463936 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6528e-388b-438e-b9c8-0515d46a916a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:14.464597 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6528e-388b-438e-b9c8-0515d46a916a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:14.464597 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6528e-388b-438e-b9c8-0515d46a916a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:14.484772 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f7a6528e-388b-438e-b9c8-0515d46a916a None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:31:14.486053 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f7a6528e-388b-438e-b9c8-0515d46a916a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:31:14.553299 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1214/2440] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:31:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:31:14.560890 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567524d4-dd86-4059-8fc0-3338f9326e25 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:14.561178 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567524d4-dd86-4059-8fc0-3338f9326e25 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:14.561484 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567524d4-dd86-4059-8fc0-3338f9326e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:14.582382 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-567524d4-dd86-4059-8fc0-3338f9326e25 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:31:14.583592 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-567524d4-dd86-4059-8fc0-3338f9326e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:31:14.596145 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1227/2441] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:31:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:31:14.596963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6528e-388b-438e-b9c8-0515d46a916a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:31:14.607186 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6528e-388b-438e-b9c8-0515d46a916a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:31:14.612205 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85acc75-c1e8-42eb-970b-12657f56195f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:31:14.612811 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85acc75-c1e8-42eb-970b-12657f56195f admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:14.613096 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85acc75-c1e8-42eb-970b-12657f56195f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:14.613679 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85acc75-c1e8-42eb-970b-12657f56195f admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:14.614432 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85acc75-c1e8-42eb-970b-12657f56195f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:31:14.614706 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85acc75-c1e8-42eb-970b-12657f56195f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:31:14.616491 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85acc75-c1e8-42eb-970b-12657f56195f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:31:14.633020 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1215/2442] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:31:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:31:14.637216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567524d4-dd86-4059-8fc0-3338f9326e25 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:31:14.647768 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567524d4-dd86-4059-8fc0-3338f9326e25 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:31:14.652865 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d41ebc-73f8-4aa4-9075-9b6747af730f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:31:14.653529 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d41ebc-73f8-4aa4-9075-9b6747af730f admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:14.653814 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d41ebc-73f8-4aa4-9075-9b6747af730f admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:14.654084 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d41ebc-73f8-4aa4-9075-9b6747af730f admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:14.654728 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d41ebc-73f8-4aa4-9075-9b6747af730f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:31:14.656658 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d41ebc-73f8-4aa4-9075-9b6747af730f admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:31:14.698724 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1228/2443] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:31:14 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:31:14.702648 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85acc75-c1e8-42eb-970b-12657f56195f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:31:14.726508 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85acc75-c1e8-42eb-970b-12657f56195f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:31:14.798639 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb9b3b-6b5d-4355-b593-1cb999b09e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:31:14.799954 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb9b3b-6b5d-4355-b593-1cb999b09e15 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:14.800958 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb9b3b-6b5d-4355-b593-1cb999b09e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:14.800958 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb9b3b-6b5d-4355-b593-1cb999b09e15 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:14.806568 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb9b3b-6b5d-4355-b593-1cb999b09e15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:31:14.806568 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb9b3b-6b5d-4355-b593-1cb999b09e15 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:31:14.843412 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1216/2444] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:31:14 2022] POST /identity/v3/users => generated 287 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:31:14.850778 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d41ebc-73f8-4aa4-9075-9b6747af730f admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:31:14.872382 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d41ebc-73f8-4aa4-9075-9b6747af730f admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:31:14.877092 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33abf3d2-9768-4ead-bb5e-e30905d0ea92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:31:14.877648 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abf3d2-9768-4ead-bb5e-e30905d0ea92 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:14.877933 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abf3d2-9768-4ead-bb5e-e30905d0ea92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:14.878213 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abf3d2-9768-4ead-bb5e-e30905d0ea92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:14.878865 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33abf3d2-9768-4ead-bb5e-e30905d0ea92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:31:14.880577 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33abf3d2-9768-4ead-bb5e-e30905d0ea92 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:31:14.896206 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1229/2445] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:31:14.899864 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb9b3b-6b5d-4355-b593-1cb999b09e15 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:31:14.917296 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb9b3b-6b5d-4355-b593-1cb999b09e15 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:31:14.920017 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb28930-9a00-4557-a0d1-4fb047909204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:31:14.920017 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb28930-9a00-4557-a0d1-4fb047909204 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:14.920017 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb28930-9a00-4557-a0d1-4fb047909204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:14.920017 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb28930-9a00-4557-a0d1-4fb047909204 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:14.920685 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb28930-9a00-4557-a0d1-4fb047909204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:31:14.922382 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb28930-9a00-4557-a0d1-4fb047909204 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:31:14.941373 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1217/2446] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:31:14.942972 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33abf3d2-9768-4ead-bb5e-e30905d0ea92 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:31:14.954752 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33abf3d2-9768-4ead-bb5e-e30905d0ea92 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:31:14.960474 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899121d8-3eaa-4095-b090-94d5e88eaad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:31:14.961149 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899121d8-3eaa-4095-b090-94d5e88eaad9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:14.961496 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899121d8-3eaa-4095-b090-94d5e88eaad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:14.961782 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899121d8-3eaa-4095-b090-94d5e88eaad9 admin admin] PATH_INFO: `/v3/projects/2f672a6f47c84495986796b8814900b2/users/a6ea21f089fa47b38fc6e9835a72ede8/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:14.977754 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899121d8-3eaa-4095-b090-94d5e88eaad9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f672a6f47c84495986796b8814900b2, user_id=a6ea21f089fa47b38fc6e9835a72ede8, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:31:14.980512 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899121d8-3eaa-4095-b090-94d5e88eaad9 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:31:14.995516 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1230/2447] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:31:14 2022] PUT /identity/v3/projects/2f672a6f47c84495986796b8814900b2/users/a6ea21f089fa47b38fc6e9835a72ede8/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:31:15.006025 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112c6d3-f327-4154-b5ba-fcaa610b71c2 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:15.006399 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112c6d3-f327-4154-b5ba-fcaa610b71c2 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:15.006752 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112c6d3-f327-4154-b5ba-fcaa610b71c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:15.043311 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-d112c6d3-f327-4154-b5ba-fcaa610b71c2 None None] MFA Rules not processed for user `a6ea21f089fa47b38fc6e9835a72ede8`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:31:15.044667 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d112c6d3-f327-4154-b5ba-fcaa610b71c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:31:15.075016 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:31:15.119960 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1218/2448] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:31:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:31:15.133041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899121d8-3eaa-4095-b090-94d5e88eaad9 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:31:15.134995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-899121d8-3eaa-4095-b090-94d5e88eaad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:31:15.211719 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:31:15.222126 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899121d8-3eaa-4095-b090-94d5e88eaad9 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:31:15.231662 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e023bacd-1952-4063-b006-4fc8e37954aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:31:15.232496 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023bacd-1952-4063-b006-4fc8e37954aa service cinder] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:15.232950 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023bacd-1952-4063-b006-4fc8e37954aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:15.233446 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023bacd-1952-4063-b006-4fc8e37954aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:15.243547 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e023bacd-1952-4063-b006-4fc8e37954aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:31:15.244511 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e023bacd-1952-4063-b006-4fc8e37954aa service cinder] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:31:15.257920 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1231/2449] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:31:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:31:16.849959 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bdcd25-41b9-4e0d-8b4b-4710d873ea9b None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:16.854615 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bdcd25-41b9-4e0d-8b4b-4710d873ea9b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:16.855944 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bdcd25-41b9-4e0d-8b4b-4710d873ea9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:16.878022 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-84bdcd25-41b9-4e0d-8b4b-4710d873ea9b None None] MFA Rules not processed for user `a6ea21f089fa47b38fc6e9835a72ede8`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:31:16.879338 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-84bdcd25-41b9-4e0d-8b4b-4710d873ea9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:31:16.890096 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1219/2450] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:31:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:31:25.172797 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e023bacd-1952-4063-b006-4fc8e37954aa service cinder] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:31:25.174266 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-e023bacd-1952-4063-b006-4fc8e37954aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:31:25.247590 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e023bacd-1952-4063-b006-4fc8e37954aa service cinder] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:31:25.251317 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a5357-3bd5-4f9d-a7a8-524f138e69df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:31:25.251934 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a5357-3bd5-4f9d-a7a8-524f138e69df service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:25.252333 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a5357-3bd5-4f9d-a7a8-524f138e69df service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:25.252610 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a5357-3bd5-4f9d-a7a8-524f138e69df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:25.255106 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6c4a5357-3bd5-4f9d-a7a8-524f138e69df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:31:25.338280 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a5357-3bd5-4f9d-a7a8-524f138e69df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:31:25.339945 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a5357-3bd5-4f9d-a7a8-524f138e69df service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:31:25.366257 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1232/2451] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:31:25 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:31:26.872827 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=18 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:31:30.144530 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639a151-9477-44e0-be3d-5280949dc0d5 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:30.145163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639a151-9477-44e0-be3d-5280949dc0d5 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:30.145163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639a151-9477-44e0-be3d-5280949dc0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:30.167323 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0639a151-9477-44e0-be3d-5280949dc0d5 None None] MFA Rules not processed for user `cedaf156c5584d62beedde598c94f322`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:31:30.168683 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0639a151-9477-44e0-be3d-5280949dc0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:31:30.228125 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1220/2452] 10.222.0.26 () {58 vars in 967 bytes} [Fri Sep 9 20:31:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:31:30.243283 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a5357-3bd5-4f9d-a7a8-524f138e69df service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:31:30.246142 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6c4a5357-3bd5-4f9d-a7a8-524f138e69df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:31:30.330539 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a5357-3bd5-4f9d-a7a8-524f138e69df service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:31:30.336185 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dc41d0-7d46-4968-817c-36f77eee65d2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb62d06f7e94e9295c54504ddabb1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:31:30.336897 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc41d0-7d46-4968-817c-36f77eee65d2 service swift] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:30.337302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc41d0-7d46-4968-817c-36f77eee65d2 service swift] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:30.337630 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc41d0-7d46-4968-817c-36f77eee65d2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:30.346134 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dc41d0-7d46-4968-817c-36f77eee65d2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:31:30.347812 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dc41d0-7d46-4968-817c-36f77eee65d2 service swift] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:31:30.356626 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1233/2453] 10.222.0.26 () {60 vars in 1384 bytes} [Fri Sep 9 20:31:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 20:31:38.565328 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9561d5b-9167-4c8b-a105-c102bdf24e10 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:38.565799 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9561d5b-9167-4c8b-a105-c102bdf24e10 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:38.565889 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9561d5b-9167-4c8b-a105-c102bdf24e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:38.584802 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a9561d5b-9167-4c8b-a105-c102bdf24e10 None None] MFA Rules not processed for user `cedaf156c5584d62beedde598c94f322`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:31:38.586037 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a9561d5b-9167-4c8b-a105-c102bdf24e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:31:38.593402 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1221/2454] 10.222.0.26 () {58 vars in 1009 bytes} [Fri Sep 9 20:31:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:31:38.605509 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dc41d0-7d46-4968-817c-36f77eee65d2 service swift] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:31:38.616244 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dc41d0-7d46-4968-817c-36f77eee65d2 service swift] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:31:38.621200 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad2405-7cac-4a47-bae8-e48018fd8fe4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb62d06f7e94e9295c54504ddabb1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:31:38.621802 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad2405-7cac-4a47-bae8-e48018fd8fe4 service swift] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:38.622078 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad2405-7cac-4a47-bae8-e48018fd8fe4 service swift] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:38.622346 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad2405-7cac-4a47-bae8-e48018fd8fe4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:38.630174 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ad2405-7cac-4a47-bae8-e48018fd8fe4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:31:38.632279 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ad2405-7cac-4a47-bae8-e48018fd8fe4 service swift] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:31:38.640088 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1234/2455] 10.222.0.26 () {60 vars in 1384 bytes} [Fri Sep 9 20:31:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 20:31:39.154281 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f65c0-7e52-4231-97aa-4cbba2f25187 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:39.154600 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f65c0-7e52-4231-97aa-4cbba2f25187 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:39.154881 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f65c0-7e52-4231-97aa-4cbba2f25187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:39.174944 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-4f6f65c0-7e52-4231-97aa-4cbba2f25187 None None] MFA Rules not processed for user `cedaf156c5584d62beedde598c94f322`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:31:39.176086 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4f6f65c0-7e52-4231-97aa-4cbba2f25187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:31:39.183978 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1222/2456] 10.222.0.26 () {58 vars in 967 bytes} [Fri Sep 9 20:31:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:31:39.196818 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad2405-7cac-4a47-bae8-e48018fd8fe4 service swift] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:31:39.207245 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad2405-7cac-4a47-bae8-e48018fd8fe4 service swift] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:31:39.212295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66ea28-9fcd-4c9b-958f-3d13510bfc1b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb62d06f7e94e9295c54504ddabb1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:31:39.212894 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66ea28-9fcd-4c9b-958f-3d13510bfc1b service swift] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:39.213172 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66ea28-9fcd-4c9b-958f-3d13510bfc1b service swift] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:39.213476 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66ea28-9fcd-4c9b-958f-3d13510bfc1b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:39.221617 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d66ea28-9fcd-4c9b-958f-3d13510bfc1b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:31:39.223201 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d66ea28-9fcd-4c9b-958f-3d13510bfc1b service swift] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:31:39.232004 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1235/2457] 10.222.0.26 () {60 vars in 1384 bytes} [Fri Sep 9 20:31:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 20:31:39.464522 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7b75e-b27f-4cc7-be39-1b45009c279d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:39.464808 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7b75e-b27f-4cc7-be39-1b45009c279d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:39.465085 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7b75e-b27f-4cc7-be39-1b45009c279d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:39.484019 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-11c7b75e-b27f-4cc7-be39-1b45009c279d None None] MFA Rules not processed for user `cedaf156c5584d62beedde598c94f322`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:31:39.485208 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-11c7b75e-b27f-4cc7-be39-1b45009c279d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:31:39.493065 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1223/2458] 10.222.0.26 () {58 vars in 1009 bytes} [Fri Sep 9 20:31:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:31:39.504839 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66ea28-9fcd-4c9b-958f-3d13510bfc1b service swift] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:31:39.511646 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:31:39.515747 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66ea28-9fcd-4c9b-958f-3d13510bfc1b service swift] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:31:39.520983 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f505d9-31c7-4262-b972-e26eca1b4ca0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb62d06f7e94e9295c54504ddabb1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:31:39.521603 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f505d9-31c7-4262-b972-e26eca1b4ca0 service swift] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:39.521992 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f505d9-31c7-4262-b972-e26eca1b4ca0 service swift] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:39.522267 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f505d9-31c7-4262-b972-e26eca1b4ca0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:39.530131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f505d9-31c7-4262-b972-e26eca1b4ca0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:31:39.531680 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f505d9-31c7-4262-b972-e26eca1b4ca0 service swift] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:31:39.540807 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1236/2459] 10.222.0.26 () {60 vars in 1384 bytes} [Fri Sep 9 20:31:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 20:31:39.871146 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ef190-baec-4710-bada-fe6a74f0abed None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:39.871608 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ef190-baec-4710-bada-fe6a74f0abed None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:39.871700 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ef190-baec-4710-bada-fe6a74f0abed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:39.891047 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-2e4ef190-baec-4710-bada-fe6a74f0abed None None] MFA Rules not processed for user `cedaf156c5584d62beedde598c94f322`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:31:39.892253 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2e4ef190-baec-4710-bada-fe6a74f0abed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:31:39.899901 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1224/2460] 10.222.0.26 () {58 vars in 967 bytes} [Fri Sep 9 20:31:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:31:39.911926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f505d9-31c7-4262-b972-e26eca1b4ca0 service swift] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:31:39.922138 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f505d9-31c7-4262-b972-e26eca1b4ca0 service swift] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:31:39.927075 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f846a82-387d-4e7e-b6aa-dc54c304d5c9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb62d06f7e94e9295c54504ddabb1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:31:39.928158 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f846a82-387d-4e7e-b6aa-dc54c304d5c9 service swift] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:39.928432 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f846a82-387d-4e7e-b6aa-dc54c304d5c9 service swift] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:39.928742 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f846a82-387d-4e7e-b6aa-dc54c304d5c9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:39.936617 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f846a82-387d-4e7e-b6aa-dc54c304d5c9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:31:39.938194 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f846a82-387d-4e7e-b6aa-dc54c304d5c9 service swift] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:31:39.946535 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1237/2461] 10.222.0.26 () {60 vars in 1384 bytes} [Fri Sep 9 20:31:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 20:31:42.025492 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ef190-baec-4710-bada-fe6a74f0abed None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:31:42.085772 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ef190-baec-4710-bada-fe6a74f0abed None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:31:42.090181 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98e0cec-ab2d-4459-89df-28ebb93575ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:31:42.090797 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e0cec-ab2d-4459-89df-28ebb93575ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:42.091076 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e0cec-ab2d-4459-89df-28ebb93575ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:42.091347 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e0cec-ab2d-4459-89df-28ebb93575ea admin admin] PATH_INFO: `/v3/users/a6ea21f089fa47b38fc6e9835a72ede8` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:42.092451 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98e0cec-ab2d-4459-89df-28ebb93575ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6ea21f089fa47b38fc6e9835a72ede8)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:31:42.094378 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98e0cec-ab2d-4459-89df-28ebb93575ea admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:31:42.143213 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b98e0cec-ab2d-4459-89df-28ebb93575ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6ea21f089fa47b38fc6e9835a72ede8'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:31:42.151117 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b98e0cec-ab2d-4459-89df-28ebb93575ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6ea21f089fa47b38fc6e9835a72ede8'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:31:42.164515 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b98e0cec-ab2d-4459-89df-28ebb93575ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6ea21f089fa47b38fc6e9835a72ede8'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:31:42.171832 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b98e0cec-ab2d-4459-89df-28ebb93575ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6ea21f089fa47b38fc6e9835a72ede8'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:31:42.172481 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b98e0cec-ab2d-4459-89df-28ebb93575ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6ea21f089fa47b38fc6e9835a72ede8'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:31:42.173944 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:31:42.179930 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1225/2462] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:31:42 2022] DELETE /identity/v3/users/a6ea21f089fa47b38fc6e9835a72ede8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:31:42.184348 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6999be4-519f-46d9-8aba-9b9b40989685 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:42.184816 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6999be4-519f-46d9-8aba-9b9b40989685 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:42.184931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6999be4-519f-46d9-8aba-9b9b40989685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:42.204065 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-b6999be4-519f-46d9-8aba-9b9b40989685 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:31:42.205301 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-b6999be4-519f-46d9-8aba-9b9b40989685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:31:42.264244 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1238/2463] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:31:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:31:42.274396 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98e0cec-ab2d-4459-89df-28ebb93575ea admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:31:42.275885 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b98e0cec-ab2d-4459-89df-28ebb93575ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:31:42.332354 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98e0cec-ab2d-4459-89df-28ebb93575ea admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:31:42.336171 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ac087-0a3b-47b0-8514-6bc5b74100ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:31:42.336778 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ac087-0a3b-47b0-8514-6bc5b74100ab service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:42.337056 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ac087-0a3b-47b0-8514-6bc5b74100ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:42.337360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ac087-0a3b-47b0-8514-6bc5b74100ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:42.346451 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790ac087-0a3b-47b0-8514-6bc5b74100ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:31:42.347926 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790ac087-0a3b-47b0-8514-6bc5b74100ab service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:31:42.357944 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1226/2464] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:31:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 20:31:42.536324 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6999be4-519f-46d9-8aba-9b9b40989685 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:31:42.537921 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-b6999be4-519f-46d9-8aba-9b9b40989685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:31:42.549780 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6999be4-519f-46d9-8aba-9b9b40989685 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:31:42.554587 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba9be4a-4fd6-463c-b70c-092156835195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:31:42.555221 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9be4a-4fd6-463c-b70c-092156835195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:31:42.555501 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9be4a-4fd6-463c-b70c-092156835195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:31:42.555776 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9be4a-4fd6-463c-b70c-092156835195 admin admin] PATH_INFO: `/v3/projects/2f672a6f47c84495986796b8814900b2` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:31:42.556892 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba9be4a-4fd6-463c-b70c-092156835195 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f672a6f47c84495986796b8814900b2)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:31:42.558816 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba9be4a-4fd6-463c-b70c-092156835195 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:31:42.611790 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-cba9be4a-4fd6-463c-b70c-092156835195 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f672a6f47c84495986796b8814900b2', 'request_id': 'req-cba9be4a-4fd6-463c-b70c-092156835195'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:31:42.620786 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-cba9be4a-4fd6-463c-b70c-092156835195 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f672a6f47c84495986796b8814900b2', 'request_id': 'req-cba9be4a-4fd6-463c-b70c-092156835195'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:31:42.626190 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-cba9be4a-4fd6-463c-b70c-092156835195 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f672a6f47c84495986796b8814900b2', 'request_id': 'req-cba9be4a-4fd6-463c-b70c-092156835195'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:31:42.637614 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-cba9be4a-4fd6-463c-b70c-092156835195 admin admin] The token cache is being invalidate because project 2f672a6f47c84495986796b8814900b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:31:42.637895 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-cba9be4a-4fd6-463c-b70c-092156835195 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:31:42.638510 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-cba9be4a-4fd6-463c-b70c-092156835195 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:31:42.639963 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1239/2465] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:31:42 2022] DELETE /identity/v3/projects/2f672a6f47c84495986796b8814900b2 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:31:52.353714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=27 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:31:52.634781 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=64,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:32:12.738124 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa8269-2dd4-466a-9d1b-9de72ec1f378 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:32:12.739118 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa8269-2dd4-466a-9d1b-9de72ec1f378 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:32:12.739228 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa8269-2dd4-466a-9d1b-9de72ec1f378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:32:12.759380 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-27aa8269-2dd4-466a-9d1b-9de72ec1f378 None None] MFA Rules not processed for user `a95746e7a84541199b43138b1a348386`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:32:12.760586 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-27aa8269-2dd4-466a-9d1b-9de72ec1f378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:32:12.814961 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1227/2466] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:32:12 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:32:22.808087 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=25 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:32:25.165483 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1feb8d0-d420-4f9c-9c16-fad75686e185 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:32:25.166013 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1feb8d0-d420-4f9c-9c16-fad75686e185 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:32:25.166155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1feb8d0-d420-4f9c-9c16-fad75686e185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:32:25.185020 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-a1feb8d0-d420-4f9c-9c16-fad75686e185 None None] MFA Rules not processed for user `db1f365ebf794941b3999adb37c48100`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:32:25.186494 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a1feb8d0-d420-4f9c-9c16-fad75686e185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:32:25.241959 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1240/2467] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:32:25 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:32:25.427399 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a652074-d657-4792-b387-1f3f3d5359eb None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:32:25.427933 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a652074-d657-4792-b387-1f3f3d5359eb None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:32:25.427933 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a652074-d657-4792-b387-1f3f3d5359eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:32:25.447198 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0a652074-d657-4792-b387-1f3f3d5359eb None None] MFA Rules not processed for user `95bc48b6541c4388b45435036a1c4525`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:32:25.448406 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0a652074-d657-4792-b387-1f3f3d5359eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:32:25.502525 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1228/2468] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:32:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:32:35.234437 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=25 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:32:35.495797 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=25 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:33:02.156546 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290758f0-073a-4ad5-a71b-c8bbb79d9a6f None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:02.157048 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290758f0-073a-4ad5-a71b-c8bbb79d9a6f None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:02.157143 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290758f0-073a-4ad5-a71b-c8bbb79d9a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:02.176387 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-290758f0-073a-4ad5-a71b-c8bbb79d9a6f None None] MFA Rules not processed for user `cedaf156c5584d62beedde598c94f322`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:33:02.177590 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-290758f0-073a-4ad5-a71b-c8bbb79d9a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:02.231252 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1241/2469] 10.222.0.26 () {58 vars in 967 bytes} [Fri Sep 9 20:33:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:33:02.276019 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a652074-d657-4792-b387-1f3f3d5359eb None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:02.277443 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0a652074-d657-4792-b387-1f3f3d5359eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:02.344365 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a652074-d657-4792-b387-1f3f3d5359eb None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:02.349985 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394b84e-5b16-40a9-9bd3-398f3d88d0a8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb62d06f7e94e9295c54504ddabb1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:02.350521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394b84e-5b16-40a9-9bd3-398f3d88d0a8 service swift] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:02.350800 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394b84e-5b16-40a9-9bd3-398f3d88d0a8 service swift] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:02.351070 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394b84e-5b16-40a9-9bd3-398f3d88d0a8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:02.358886 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394b84e-5b16-40a9-9bd3-398f3d88d0a8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:02.360432 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394b84e-5b16-40a9-9bd3-398f3d88d0a8 service swift] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:02.368620 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1229/2470] 10.222.0.26 () {60 vars in 1384 bytes} [Fri Sep 9 20:33:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 20:33:05.671215 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290758f0-073a-4ad5-a71b-c8bbb79d9a6f None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:05.672666 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-290758f0-073a-4ad5-a71b-c8bbb79d9a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:05.745254 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290758f0-073a-4ad5-a71b-c8bbb79d9a6f None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:05.749230 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88138811-2500-45bb-a76b-f92c746ccc73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:05.749891 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88138811-2500-45bb-a76b-f92c746ccc73 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:05.750173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88138811-2500-45bb-a76b-f92c746ccc73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:05.750445 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88138811-2500-45bb-a76b-f92c746ccc73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:05.752464 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-88138811-2500-45bb-a76b-f92c746ccc73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:05.849769 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88138811-2500-45bb-a76b-f92c746ccc73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:05.851231 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88138811-2500-45bb-a76b-f92c746ccc73 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:05.864127 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1242/2471] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:33:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:33:07.081446 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394b84e-5b16-40a9-9bd3-398f3d88d0a8 service swift] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:07.094412 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394b84e-5b16-40a9-9bd3-398f3d88d0a8 service swift] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:07.098272 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9941c2f7-0a07-4d65-b293-fbecb7a088be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:07.098996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9941c2f7-0a07-4d65-b293-fbecb7a088be service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:07.100214 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9941c2f7-0a07-4d65-b293-fbecb7a088be service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:07.100593 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9941c2f7-0a07-4d65-b293-fbecb7a088be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:07.107556 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9941c2f7-0a07-4d65-b293-fbecb7a088be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:07.123208 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9941c2f7-0a07-4d65-b293-fbecb7a088be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:07.124709 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9941c2f7-0a07-4d65-b293-fbecb7a088be service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:07.136524 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1230/2472] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:33:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:33:09.481720 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88138811-2500-45bb-a76b-f92c746ccc73 service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:09.483850 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-88138811-2500-45bb-a76b-f92c746ccc73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:09.535840 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:33:09.553154 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88138811-2500-45bb-a76b-f92c746ccc73 service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:09.557820 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc25df5-4985-4b16-965a-b6c09ed75ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:09.558465 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc25df5-4985-4b16-965a-b6c09ed75ad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:09.558747 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc25df5-4985-4b16-965a-b6c09ed75ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:09.559023 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc25df5-4985-4b16-965a-b6c09ed75ad0 admin admin] PATH_INFO: `/v3/users/ec90aaf1a60746f69e5478987ab86bc1` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:09.560188 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc25df5-4985-4b16-965a-b6c09ed75ad0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec90aaf1a60746f69e5478987ab86bc1)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:09.565066 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc25df5-4985-4b16-965a-b6c09ed75ad0 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:09.616485 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-bfc25df5-4985-4b16-965a-b6c09ed75ad0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec90aaf1a60746f69e5478987ab86bc1'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:09.631954 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-bfc25df5-4985-4b16-965a-b6c09ed75ad0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec90aaf1a60746f69e5478987ab86bc1'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:09.639797 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-bfc25df5-4985-4b16-965a-b6c09ed75ad0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec90aaf1a60746f69e5478987ab86bc1'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:09.648160 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-bfc25df5-4985-4b16-965a-b6c09ed75ad0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec90aaf1a60746f69e5478987ab86bc1'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:09.648835 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-bfc25df5-4985-4b16-965a-b6c09ed75ad0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec90aaf1a60746f69e5478987ab86bc1'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:09.656334 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1243/2473] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:33:09 2022] DELETE /identity/v3/users/ec90aaf1a60746f69e5478987ab86bc1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:33:09.834758 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9941c2f7-0a07-4d65-b293-fbecb7a088be service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:09.847107 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9941c2f7-0a07-4d65-b293-fbecb7a088be service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:09.904276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c9290-d426-4146-9308-31751c62e938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:09.904912 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c9290-d426-4146-9308-31751c62e938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:09.905189 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c9290-d426-4146-9308-31751c62e938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:09.905490 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c9290-d426-4146-9308-31751c62e938 admin admin] PATH_INFO: `/v3/projects/da5cdb6c865f4345be2de638e6ded3c6` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:09.906586 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72c9290-d426-4146-9308-31751c62e938 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da5cdb6c865f4345be2de638e6ded3c6)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:09.908460 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72c9290-d426-4146-9308-31751c62e938 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:09.953488 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f72c9290-d426-4146-9308-31751c62e938 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da5cdb6c865f4345be2de638e6ded3c6', 'request_id': 'req-f72c9290-d426-4146-9308-31751c62e938'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:09.959085 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f72c9290-d426-4146-9308-31751c62e938 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da5cdb6c865f4345be2de638e6ded3c6', 'request_id': 'req-f72c9290-d426-4146-9308-31751c62e938'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:09.965358 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f72c9290-d426-4146-9308-31751c62e938 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da5cdb6c865f4345be2de638e6ded3c6', 'request_id': 'req-f72c9290-d426-4146-9308-31751c62e938'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:09.981193 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f72c9290-d426-4146-9308-31751c62e938 admin admin] The token cache is being invalidate because project da5cdb6c865f4345be2de638e6ded3c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:33:09.981542 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f72c9290-d426-4146-9308-31751c62e938 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:09.982184 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f72c9290-d426-4146-9308-31751c62e938 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:09.983662 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1231/2474] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:33:09 2022] DELETE /identity/v3/projects/da5cdb6c865f4345be2de638e6ded3c6 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:33:09.988490 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc25df5-4985-4b16-965a-b6c09ed75ad0 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:09.989931 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-bfc25df5-4985-4b16-965a-b6c09ed75ad0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:10.002815 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc25df5-4985-4b16-965a-b6c09ed75ad0 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:10.057201 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a635fe4-1110-4589-9d37-57bfab059150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:10.057842 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a635fe4-1110-4589-9d37-57bfab059150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:10.058118 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a635fe4-1110-4589-9d37-57bfab059150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:10.058389 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a635fe4-1110-4589-9d37-57bfab059150 admin admin] PATH_INFO: `/v3/users/9a27aba7d18942fe80504abd4006f5e6` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:10.059495 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a635fe4-1110-4589-9d37-57bfab059150 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a27aba7d18942fe80504abd4006f5e6)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:10.061428 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a635fe4-1110-4589-9d37-57bfab059150 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:10.107308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-2a635fe4-1110-4589-9d37-57bfab059150 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a27aba7d18942fe80504abd4006f5e6'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:10.120110 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-2a635fe4-1110-4589-9d37-57bfab059150 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a27aba7d18942fe80504abd4006f5e6'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:10.122086 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:33:10.129189 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-2a635fe4-1110-4589-9d37-57bfab059150 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a27aba7d18942fe80504abd4006f5e6'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:10.136605 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-2a635fe4-1110-4589-9d37-57bfab059150 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a27aba7d18942fe80504abd4006f5e6'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:10.137397 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-2a635fe4-1110-4589-9d37-57bfab059150 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a27aba7d18942fe80504abd4006f5e6'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:10.144271 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1244/2475] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:33:09 2022] DELETE /identity/v3/users/9a27aba7d18942fe80504abd4006f5e6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:33:10.313096 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c9290-d426-4146-9308-31751c62e938 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:10.314715 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f72c9290-d426-4146-9308-31751c62e938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:10.329125 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c9290-d426-4146-9308-31751c62e938 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:10.333846 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:33:10.390619 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7fc4c2-e4bc-4f84-b54c-d45ede390720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:10.391181 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7fc4c2-e4bc-4f84-b54c-d45ede390720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:10.391459 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7fc4c2-e4bc-4f84-b54c-d45ede390720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:10.392227 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7fc4c2-e4bc-4f84-b54c-d45ede390720 admin admin] PATH_INFO: `/v3/projects/33294a040dca4909a3cbeda469245c75` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:10.393388 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7fc4c2-e4bc-4f84-b54c-d45ede390720 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33294a040dca4909a3cbeda469245c75)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:10.395262 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7fc4c2-e4bc-4f84-b54c-d45ede390720 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:10.440436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2e7fc4c2-e4bc-4f84-b54c-d45ede390720 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33294a040dca4909a3cbeda469245c75', 'request_id': 'req-2e7fc4c2-e4bc-4f84-b54c-d45ede390720'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:10.446050 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2e7fc4c2-e4bc-4f84-b54c-d45ede390720 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33294a040dca4909a3cbeda469245c75', 'request_id': 'req-2e7fc4c2-e4bc-4f84-b54c-d45ede390720'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:10.452249 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2e7fc4c2-e4bc-4f84-b54c-d45ede390720 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33294a040dca4909a3cbeda469245c75', 'request_id': 'req-2e7fc4c2-e4bc-4f84-b54c-d45ede390720'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:10.465010 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2e7fc4c2-e4bc-4f84-b54c-d45ede390720 admin admin] The token cache is being invalidate because project 33294a040dca4909a3cbeda469245c75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:33:10.465336 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2e7fc4c2-e4bc-4f84-b54c-d45ede390720 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:10.465950 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2e7fc4c2-e4bc-4f84-b54c-d45ede390720 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:10.467430 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1232/2476] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:33:10 2022] DELETE /identity/v3/projects/33294a040dca4909a3cbeda469245c75 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:33:15.506637 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a635fe4-1110-4589-9d37-57bfab059150 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:15.508049 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-2a635fe4-1110-4589-9d37-57bfab059150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:15.564770 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a635fe4-1110-4589-9d37-57bfab059150 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:15.611218 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c065a1-a6bc-4363-a4d5-31d9ec1625d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:15.612063 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c065a1-a6bc-4363-a4d5-31d9ec1625d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:15.612337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c065a1-a6bc-4363-a4d5-31d9ec1625d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:15.612602 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c065a1-a6bc-4363-a4d5-31d9ec1625d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:15.614317 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-10c065a1-a6bc-4363-a4d5-31d9ec1625d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:15.674642 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c065a1-a6bc-4363-a4d5-31d9ec1625d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:15.676259 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c065a1-a6bc-4363-a4d5-31d9ec1625d5 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:15.686024 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1245/2477] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:33:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:33:17.413863 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7fc4c2-e4bc-4f84-b54c-d45ede390720 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:17.415442 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2e7fc4c2-e4bc-4f84-b54c-d45ede390720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:17.499134 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7fc4c2-e4bc-4f84-b54c-d45ede390720 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:17.506630 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3431e-18f8-4f4b-b8c6-a52d66131f34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:17.507527 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3431e-18f8-4f4b-b8c6-a52d66131f34 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:17.508124 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3431e-18f8-4f4b-b8c6-a52d66131f34 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:17.508598 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3431e-18f8-4f4b-b8c6-a52d66131f34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:17.510836 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ecb3431e-18f8-4f4b-b8c6-a52d66131f34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:17.551458 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:33:17.583215 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb3431e-18f8-4f4b-b8c6-a52d66131f34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:17.585223 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb3431e-18f8-4f4b-b8c6-a52d66131f34 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:17.595240 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1233/2478] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:33:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:33:18.867812 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c065a1-a6bc-4363-a4d5-31d9ec1625d5 service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:18.869331 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-10c065a1-a6bc-4363-a4d5-31d9ec1625d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:18.881937 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c065a1-a6bc-4363-a4d5-31d9ec1625d5 service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:18.886540 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9083ec-4531-4ebe-80ec-f3206af557be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:18.887209 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9083ec-4531-4ebe-80ec-f3206af557be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:18.887492 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9083ec-4531-4ebe-80ec-f3206af557be admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:18.887767 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9083ec-4531-4ebe-80ec-f3206af557be admin admin] PATH_INFO: `/v3/users/a95746e7a84541199b43138b1a348386` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:18.888894 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9083ec-4531-4ebe-80ec-f3206af557be admin admin] RBAC: Authorizing `identity:delete_user(user_id=a95746e7a84541199b43138b1a348386)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:18.890859 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9083ec-4531-4ebe-80ec-f3206af557be admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:18.941165 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:33:18.943132 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-7f9083ec-4531-4ebe-80ec-f3206af557be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a95746e7a84541199b43138b1a348386'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:18.957455 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-7f9083ec-4531-4ebe-80ec-f3206af557be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a95746e7a84541199b43138b1a348386'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:18.964476 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-7f9083ec-4531-4ebe-80ec-f3206af557be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a95746e7a84541199b43138b1a348386'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:18.972765 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-7f9083ec-4531-4ebe-80ec-f3206af557be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a95746e7a84541199b43138b1a348386'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:18.973193 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-7f9083ec-4531-4ebe-80ec-f3206af557be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a95746e7a84541199b43138b1a348386'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:18.979295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1246/2479] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:33:18 2022] DELETE /identity/v3/users/a95746e7a84541199b43138b1a348386 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:33:19.093372 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3431e-18f8-4f4b-b8c6-a52d66131f34 service nova] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:19.109124 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3431e-18f8-4f4b-b8c6-a52d66131f34 service nova] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:19.179036 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9083ec-4531-4ebe-80ec-f3206af557be admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:19.180435 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7f9083ec-4531-4ebe-80ec-f3206af557be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:19.212764 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f91827-fb77-4eb1-885a-bc16efc7aa1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:19.213525 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f91827-fb77-4eb1-885a-bc16efc7aa1f service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:19.213872 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f91827-fb77-4eb1-885a-bc16efc7aa1f service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:19.214452 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f91827-fb77-4eb1-885a-bc16efc7aa1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:19.217455 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-00f91827-fb77-4eb1-885a-bc16efc7aa1f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:19.264246 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9083ec-4531-4ebe-80ec-f3206af557be admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:19.270227 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d6677-61ef-4a71-9f8c-6b2e6a39ef42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:19.270979 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d6677-61ef-4a71-9f8c-6b2e6a39ef42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:19.271329 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d6677-61ef-4a71-9f8c-6b2e6a39ef42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:19.271680 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d6677-61ef-4a71-9f8c-6b2e6a39ef42 admin admin] PATH_INFO: `/v3/projects/f07defd609684351b4b7d0d110212a7c` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:19.272805 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d6677-61ef-4a71-9f8c-6b2e6a39ef42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f07defd609684351b4b7d0d110212a7c)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:19.274908 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d6677-61ef-4a71-9f8c-6b2e6a39ef42 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:19.289547 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f91827-fb77-4eb1-885a-bc16efc7aa1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:19.291470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f91827-fb77-4eb1-885a-bc16efc7aa1f service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:19.311509 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1234/2480] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:33:19 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:33:19.340548 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-f04d6677-61ef-4a71-9f8c-6b2e6a39ef42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f07defd609684351b4b7d0d110212a7c', 'request_id': 'req-f04d6677-61ef-4a71-9f8c-6b2e6a39ef42'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:19.350181 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-f04d6677-61ef-4a71-9f8c-6b2e6a39ef42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f07defd609684351b4b7d0d110212a7c', 'request_id': 'req-f04d6677-61ef-4a71-9f8c-6b2e6a39ef42'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:19.357719 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-f04d6677-61ef-4a71-9f8c-6b2e6a39ef42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f07defd609684351b4b7d0d110212a7c', 'request_id': 'req-f04d6677-61ef-4a71-9f8c-6b2e6a39ef42'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:19.374859 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-f04d6677-61ef-4a71-9f8c-6b2e6a39ef42 admin admin] The token cache is being invalidate because project f07defd609684351b4b7d0d110212a7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:33:19.375140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-f04d6677-61ef-4a71-9f8c-6b2e6a39ef42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:19.375918 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-f04d6677-61ef-4a71-9f8c-6b2e6a39ef42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:19.377868 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1247/2481] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:33:19 2022] DELETE /identity/v3/projects/f07defd609684351b4b7d0d110212a7c => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:33:19.382991 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b981a-d81b-43ff-909d-9cc65f58b047 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:19.383324 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b981a-d81b-43ff-909d-9cc65f58b047 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:19.383609 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b981a-d81b-43ff-909d-9cc65f58b047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:19.407072 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-536b981a-d81b-43ff-909d-9cc65f58b047 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:33:19.409293 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-536b981a-d81b-43ff-909d-9cc65f58b047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:19.477329 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1235/2482] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:33:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:33:19.484523 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4c8f0-0e57-4570-8c0a-e7a7bc727cbb None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:19.484813 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4c8f0-0e57-4570-8c0a-e7a7bc727cbb None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:19.485086 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4c8f0-0e57-4570-8c0a-e7a7bc727cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:19.506343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-51a4c8f0-0e57-4570-8c0a-e7a7bc727cbb None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:33:19.507556 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-51a4c8f0-0e57-4570-8c0a-e7a7bc727cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:19.515949 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1248/2483] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:33:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:33:19.520331 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b981a-d81b-43ff-909d-9cc65f58b047 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:19.530625 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b981a-d81b-43ff-909d-9cc65f58b047 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:19.536497 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55baf94f-9839-4d2b-92ab-e88d37b19862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:19.537345 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55baf94f-9839-4d2b-92ab-e88d37b19862 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:19.537724 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55baf94f-9839-4d2b-92ab-e88d37b19862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:19.538386 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55baf94f-9839-4d2b-92ab-e88d37b19862 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:19.539159 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55baf94f-9839-4d2b-92ab-e88d37b19862 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:33:19.539485 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55baf94f-9839-4d2b-92ab-e88d37b19862 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:19.541198 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55baf94f-9839-4d2b-92ab-e88d37b19862 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:19.549492 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:33:19.555089 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1236/2484] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:33:19.559874 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a4c8f0-0e57-4570-8c0a-e7a7bc727cbb None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:19.570107 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a4c8f0-0e57-4570-8c0a-e7a7bc727cbb None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:19.575302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aedbad7-8803-4a22-a63f-967319f0fdc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:19.577674 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aedbad7-8803-4a22-a63f-967319f0fdc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:19.577996 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aedbad7-8803-4a22-a63f-967319f0fdc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:19.578275 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aedbad7-8803-4a22-a63f-967319f0fdc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:19.579598 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aedbad7-8803-4a22-a63f-967319f0fdc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:19.581535 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aedbad7-8803-4a22-a63f-967319f0fdc6 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:19.614497 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1249/2485] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:33:19 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:33:19.619025 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55baf94f-9839-4d2b-92ab-e88d37b19862 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:19.629805 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55baf94f-9839-4d2b-92ab-e88d37b19862 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:19.689952 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cfc786-6685-45c6-a2ed-ae06960752ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:19.690558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cfc786-6685-45c6-a2ed-ae06960752ad admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:19.690836 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cfc786-6685-45c6-a2ed-ae06960752ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:19.691106 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cfc786-6685-45c6-a2ed-ae06960752ad admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:19.691741 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cfc786-6685-45c6-a2ed-ae06960752ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:19.693816 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cfc786-6685-45c6-a2ed-ae06960752ad admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:19.717597 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1237/2486] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:33:19 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:33:19.722537 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aedbad7-8803-4a22-a63f-967319f0fdc6 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:19.733635 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aedbad7-8803-4a22-a63f-967319f0fdc6 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:19.738986 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab70e31f-d21c-4c65-a984-defec01426ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:19.740391 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70e31f-d21c-4c65-a984-defec01426ee admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:19.740885 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70e31f-d21c-4c65-a984-defec01426ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:19.741233 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70e31f-d21c-4c65-a984-defec01426ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:19.741983 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab70e31f-d21c-4c65-a984-defec01426ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:19.743670 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab70e31f-d21c-4c65-a984-defec01426ee admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:19.761368 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1250/2487] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:33:19.764846 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cfc786-6685-45c6-a2ed-ae06960752ad admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:19.775460 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cfc786-6685-45c6-a2ed-ae06960752ad admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:19.780148 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cf6d3a-1a48-4aa4-aeb4-15c7108b896a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:19.780758 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf6d3a-1a48-4aa4-aeb4-15c7108b896a admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:19.781041 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf6d3a-1a48-4aa4-aeb4-15c7108b896a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:19.781351 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf6d3a-1a48-4aa4-aeb4-15c7108b896a admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:19.790101 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cf6d3a-1a48-4aa4-aeb4-15c7108b896a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:19.791745 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cf6d3a-1a48-4aa4-aeb4-15c7108b896a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:19.804751 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1238/2488] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:33:19.808776 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab70e31f-d21c-4c65-a984-defec01426ee admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:19.818763 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab70e31f-d21c-4c65-a984-defec01426ee admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:19.823335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e8171-a064-478c-8983-88e0551f244c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:19.823995 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e8171-a064-478c-8983-88e0551f244c admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:19.824263 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e8171-a064-478c-8983-88e0551f244c admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:19.824538 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e8171-a064-478c-8983-88e0551f244c admin admin] PATH_INFO: `/v3/projects/624ed22fc46047e1a842200db3388b63/users/f0eca860c013457ea79e4633f3e6d895/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:19.835422 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:33:19.839784 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444e8171-a064-478c-8983-88e0551f244c admin admin] RBAC: Authorizing `identity:create_grant(project_id=624ed22fc46047e1a842200db3388b63, user_id=f0eca860c013457ea79e4633f3e6d895, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:19.842751 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444e8171-a064-478c-8983-88e0551f244c admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:19.856553 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1251/2489] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:33:19 2022] PUT /identity/v3/projects/624ed22fc46047e1a842200db3388b63/users/f0eca860c013457ea79e4633f3e6d895/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:33:19.862474 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb231b-cc9e-4206-92de-23c2e7b84f9b None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:19.862766 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb231b-cc9e-4206-92de-23c2e7b84f9b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:19.863044 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb231b-cc9e-4206-92de-23c2e7b84f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:19.881967 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-95eb231b-cc9e-4206-92de-23c2e7b84f9b None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:33:19.883170 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-95eb231b-cc9e-4206-92de-23c2e7b84f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:19.941467 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1239/2490] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:33:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:33:19.950982 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e8171-a064-478c-8983-88e0551f244c admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:19.952717 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-444e8171-a064-478c-8983-88e0551f244c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:20.008890 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e8171-a064-478c-8983-88e0551f244c admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:20.012514 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca43e8d-9a60-45ed-a0d6-0bfdb194d8ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:20.013112 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca43e8d-9a60-45ed-a0d6-0bfdb194d8ee service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:20.013428 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca43e8d-9a60-45ed-a0d6-0bfdb194d8ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:20.013706 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca43e8d-9a60-45ed-a0d6-0bfdb194d8ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:20.023131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca43e8d-9a60-45ed-a0d6-0bfdb194d8ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:20.024556 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca43e8d-9a60-45ed-a0d6-0bfdb194d8ee service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:20.033784 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1252/2491] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:33:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 20:33:24.155666 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae46523-814d-4213-b8cc-4e41d8f834cf None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:24.156142 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae46523-814d-4213-b8cc-4e41d8f834cf None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:24.156248 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae46523-814d-4213-b8cc-4e41d8f834cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:24.189192 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0ae46523-814d-4213-b8cc-4e41d8f834cf None None] MFA Rules not processed for user `f0eca860c013457ea79e4633f3e6d895`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:33:24.190547 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0ae46523-814d-4213-b8cc-4e41d8f834cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:24.275035 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1240/2492] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:33:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:33:24.290422 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca43e8d-9a60-45ed-a0d6-0bfdb194d8ee service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:24.291882 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-cca43e8d-9a60-45ed-a0d6-0bfdb194d8ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:24.372640 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca43e8d-9a60-45ed-a0d6-0bfdb194d8ee service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:24.378289 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b28dbc6-61ab-489b-b137-46f416d8095c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:24.378914 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b28dbc6-61ab-489b-b137-46f416d8095c service nova] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:24.379193 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b28dbc6-61ab-489b-b137-46f416d8095c service nova] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:24.379466 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b28dbc6-61ab-489b-b137-46f416d8095c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:24.388020 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b28dbc6-61ab-489b-b137-46f416d8095c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:24.389581 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b28dbc6-61ab-489b-b137-46f416d8095c service nova] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:24.399148 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1253/2493] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:33:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:33:25.834112 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d3a70-4136-467f-b5cd-936c85d1d5e3 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:25.834650 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d3a70-4136-467f-b5cd-936c85d1d5e3 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:25.834876 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d3a70-4136-467f-b5cd-936c85d1d5e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:25.851117 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:33:25.857183 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-005d3a70-4136-467f-b5cd-936c85d1d5e3 None None] MFA Rules not processed for user `f0eca860c013457ea79e4633f3e6d895`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:33:25.860943 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-005d3a70-4136-467f-b5cd-936c85d1d5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:25.868902 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1241/2494] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:33:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:33:26.542586 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b28dbc6-61ab-489b-b137-46f416d8095c service nova] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:26.544176 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-1b28dbc6-61ab-489b-b137-46f416d8095c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:26.557370 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b28dbc6-61ab-489b-b137-46f416d8095c service nova] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:26.561281 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56cc16-46a5-4869-bd64-2e9c87e2f17c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:26.561897 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56cc16-46a5-4869-bd64-2e9c87e2f17c service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:26.562179 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56cc16-46a5-4869-bd64-2e9c87e2f17c service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:26.562449 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56cc16-46a5-4869-bd64-2e9c87e2f17c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:26.564539 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-ef56cc16-46a5-4869-bd64-2e9c87e2f17c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:26.577212 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef56cc16-46a5-4869-bd64-2e9c87e2f17c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:26.578746 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef56cc16-46a5-4869-bd64-2e9c87e2f17c service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:26.588038 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1254/2495] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:33:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:33:26.961576 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d3a70-4136-467f-b5cd-936c85d1d5e3 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:26.963436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-005d3a70-4136-467f-b5cd-936c85d1d5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:27.061983 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d3a70-4136-467f-b5cd-936c85d1d5e3 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:27.067891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df606c90-0973-45b1-9736-33eaf53b3ace service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26d24cf0a21d491082b0c2f11e749446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:27.068641 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df606c90-0973-45b1-9736-33eaf53b3ace service placement] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:27.069028 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df606c90-0973-45b1-9736-33eaf53b3ace service placement] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:27.069445 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df606c90-0973-45b1-9736-33eaf53b3ace service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:27.072268 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-df606c90-0973-45b1-9736-33eaf53b3ace service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:27.088547 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df606c90-0973-45b1-9736-33eaf53b3ace service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:27.090712 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df606c90-0973-45b1-9736-33eaf53b3ace service placement] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:27.104967 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1242/2496] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:33:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:33:29.242202 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56cc16-46a5-4869-bd64-2e9c87e2f17c service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:29.254464 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56cc16-46a5-4869-bd64-2e9c87e2f17c service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:29.260662 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380432c-56e9-401b-b4a9-8a5c33d74a86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:29.261364 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380432c-56e9-401b-b4a9-8a5c33d74a86 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:29.262037 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380432c-56e9-401b-b4a9-8a5c33d74a86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:29.262037 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380432c-56e9-401b-b4a9-8a5c33d74a86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:29.264546 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-3380432c-56e9-401b-b4a9-8a5c33d74a86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:29.278628 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3380432c-56e9-401b-b4a9-8a5c33d74a86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:29.280107 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3380432c-56e9-401b-b4a9-8a5c33d74a86 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:29.301546 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1255/2497] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:33:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:33:32.549176 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136439e-0788-482e-86b8-717e880e1c09 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:32.549685 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136439e-0788-482e-86b8-717e880e1c09 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:32.549861 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136439e-0788-482e-86b8-717e880e1c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:32.570709 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-9136439e-0788-482e-86b8-717e880e1c09 None None] MFA Rules not processed for user `cedaf156c5584d62beedde598c94f322`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:33:32.571842 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9136439e-0788-482e-86b8-717e880e1c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:32.627441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1243/2498] 10.222.0.26 () {58 vars in 1009 bytes} [Fri Sep 9 20:33:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:33:32.639723 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380432c-56e9-401b-b4a9-8a5c33d74a86 service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:32.641264 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-3380432c-56e9-401b-b4a9-8a5c33d74a86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:32.711189 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:33:32.733973 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380432c-56e9-401b-b4a9-8a5c33d74a86 service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:32.740130 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc46b3-dbe4-4108-b459-30310b121cba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb62d06f7e94e9295c54504ddabb1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:32.740746 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc46b3-dbe4-4108-b459-30310b121cba service swift] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:32.741031 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc46b3-dbe4-4108-b459-30310b121cba service swift] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:32.741372 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc46b3-dbe4-4108-b459-30310b121cba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:32.751391 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc46b3-dbe4-4108-b459-30310b121cba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:32.753021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc46b3-dbe4-4108-b459-30310b121cba service swift] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:32.762080 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1256/2499] 10.222.0.26 () {60 vars in 1384 bytes} [Fri Sep 9 20:33:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 20:33:39.146278 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9136439e-0788-482e-86b8-717e880e1c09 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:39.148318 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9136439e-0788-482e-86b8-717e880e1c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:39.166888 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9136439e-0788-482e-86b8-717e880e1c09 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:39.174930 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e367c832-30ce-4d12-ba13-c95e3b89cf18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:39.176050 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367c832-30ce-4d12-ba13-c95e3b89cf18 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:39.176587 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367c832-30ce-4d12-ba13-c95e3b89cf18 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:39.177079 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367c832-30ce-4d12-ba13-c95e3b89cf18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:39.180310 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e367c832-30ce-4d12-ba13-c95e3b89cf18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:39.197397 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e367c832-30ce-4d12-ba13-c95e3b89cf18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:39.199824 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e367c832-30ce-4d12-ba13-c95e3b89cf18 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:39.216883 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1244/2500] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:33:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 20:33:42.758553 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=9 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:33:47.805011 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92acf8-cc42-4559-8a59-09e29de96792 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:47.805732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92acf8-cc42-4559-8a59-09e29de96792 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:47.805732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92acf8-cc42-4559-8a59-09e29de96792 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:47.806951 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1257/2501] 10.222.0.26 () {54 vars in 921 bytes} [Fri Sep 9 20:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:33:47.813009 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc55653-d471-49f7-996b-107688f60d84 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:47.813486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc55653-d471-49f7-996b-107688f60d84 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:47.813706 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc55653-d471-49f7-996b-107688f60d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:47.834630 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0cc55653-d471-49f7-996b-107688f60d84 None None] MFA Rules not processed for user `87ece6128f9245f7b49f3e7a9faaed22`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:33:47.835845 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0cc55653-d471-49f7-996b-107688f60d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:47.851934 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1245/2502] 10.222.0.26 () {58 vars in 1018 bytes} [Fri Sep 9 20:33:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:33:47.863410 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be92acf8-cc42-4559-8a59-09e29de96792 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:47.865175 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-be92acf8-cc42-4559-8a59-09e29de96792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:47.927808 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be92acf8-cc42-4559-8a59-09e29de96792 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:47.931392 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609595b2-9f15-40a1-a4d4-6dda052b3d79 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5298486310004b63b51116dbc747e31d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:47.932072 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609595b2-9f15-40a1-a4d4-6dda052b3d79 service glance] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:47.932356 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609595b2-9f15-40a1-a4d4-6dda052b3d79 service glance] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:47.932631 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609595b2-9f15-40a1-a4d4-6dda052b3d79 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:47.942910 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609595b2-9f15-40a1-a4d4-6dda052b3d79 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:47.944323 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609595b2-9f15-40a1-a4d4-6dda052b3d79 service glance] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:47.955368 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1258/2503] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:33:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:33:48.023109 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df4c3d-3fac-467f-8ac4-c438abddd468 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:48.023395 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df4c3d-3fac-467f-8ac4-c438abddd468 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:48.023665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df4c3d-3fac-467f-8ac4-c438abddd468 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:48.025066 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1246/2504] 10.222.0.26 () {54 vars in 921 bytes} [Fri Sep 9 20:33:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:33:48.029837 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a2483c-dbf9-4bf6-b96e-245db990588a None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:48.030125 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a2483c-dbf9-4bf6-b96e-245db990588a None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:48.030418 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a2483c-dbf9-4bf6-b96e-245db990588a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:48.050353 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-a8a2483c-dbf9-4bf6-b96e-245db990588a None None] MFA Rules not processed for user `ee5c6e51f2694bf5ae7ea097cf33fb3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:33:48.051545 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a8a2483c-dbf9-4bf6-b96e-245db990588a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:48.058207 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1259/2505] 10.222.0.26 () {58 vars in 1018 bytes} [Fri Sep 9 20:33:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:33:48.069668 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64df4c3d-3fac-467f-8ac4-c438abddd468 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:48.080764 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64df4c3d-3fac-467f-8ac4-c438abddd468 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:48.084232 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d48b62-6823-4329-bca7-151366a8c1cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:48.084817 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d48b62-6823-4329-bca7-151366a8c1cc service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:48.085090 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d48b62-6823-4329-bca7-151366a8c1cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:48.085389 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d48b62-6823-4329-bca7-151366a8c1cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:48.092296 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d48b62-6823-4329-bca7-151366a8c1cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:48.094290 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d48b62-6823-4329-bca7-151366a8c1cc service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:48.102000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1247/2506] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:33:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:33:48.181787 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b279783-b038-4e03-b358-e1906dad5cc4 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:48.182075 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b279783-b038-4e03-b358-e1906dad5cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:48.182348 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b279783-b038-4e03-b358-e1906dad5cc4 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:48.183689 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1260/2507] 10.222.0.26 () {54 vars in 921 bytes} [Fri Sep 9 20:33:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:33:48.188365 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b51d6-8658-4eb9-a817-7b1e871bb678 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:48.188655 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b51d6-8658-4eb9-a817-7b1e871bb678 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:48.188931 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b51d6-8658-4eb9-a817-7b1e871bb678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:48.208363 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e09b51d6-8658-4eb9-a817-7b1e871bb678 None None] MFA Rules not processed for user `87ece6128f9245f7b49f3e7a9faaed22`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:33:48.209577 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e09b51d6-8658-4eb9-a817-7b1e871bb678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:48.219309 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1248/2508] 10.222.0.26 () {58 vars in 1018 bytes} [Fri Sep 9 20:33:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:33:48.229345 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b279783-b038-4e03-b358-e1906dad5cc4 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:48.240689 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b279783-b038-4e03-b358-e1906dad5cc4 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:48.244245 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d4049e-0356-4f4a-9e6c-02b53a8d9ebd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:48.244845 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4049e-0356-4f4a-9e6c-02b53a8d9ebd service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:48.245121 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4049e-0356-4f4a-9e6c-02b53a8d9ebd service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:48.245417 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4049e-0356-4f4a-9e6c-02b53a8d9ebd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:48.255693 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d4049e-0356-4f4a-9e6c-02b53a8d9ebd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:48.257104 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d4049e-0356-4f4a-9e6c-02b53a8d9ebd service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:48.267865 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1261/2509] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:33:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 20:33:49.965470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b79bd2-053b-43f2-992b-0ed5b6b94561 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:49.965933 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b79bd2-053b-43f2-992b-0ed5b6b94561 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:49.966028 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b79bd2-053b-43f2-992b-0ed5b6b94561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:49.984782 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-51b79bd2-053b-43f2-992b-0ed5b6b94561 None None] MFA Rules not processed for user `cedaf156c5584d62beedde598c94f322`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:33:49.986049 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-51b79bd2-053b-43f2-992b-0ed5b6b94561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:49.993493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1249/2510] 10.222.0.26 () {58 vars in 1009 bytes} [Fri Sep 9 20:33:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:33:50.006339 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d4049e-0356-4f4a-9e6c-02b53a8d9ebd service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:50.018202 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d4049e-0356-4f4a-9e6c-02b53a8d9ebd service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:50.023857 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36785e77-dfcb-4088-8ded-ef72942e72b5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb62d06f7e94e9295c54504ddabb1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:50.024718 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36785e77-dfcb-4088-8ded-ef72942e72b5 service swift] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:50.025167 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36785e77-dfcb-4088-8ded-ef72942e72b5 service swift] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:50.025656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36785e77-dfcb-4088-8ded-ef72942e72b5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:50.035083 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36785e77-dfcb-4088-8ded-ef72942e72b5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:50.036739 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36785e77-dfcb-4088-8ded-ef72942e72b5 service swift] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:50.055134 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1262/2511] 10.222.0.26 () {60 vars in 1384 bytes} [Fri Sep 9 20:33:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 20:33:50.997610 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b79bd2-053b-43f2-992b-0ed5b6b94561 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:51.007565 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b79bd2-053b-43f2-992b-0ed5b6b94561 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:51.011656 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bcb029-0627-4aa8-b9da-8d54a6e67a94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:51.013079 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcb029-0627-4aa8-b9da-8d54a6e67a94 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:51.013575 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcb029-0627-4aa8-b9da-8d54a6e67a94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:51.014038 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcb029-0627-4aa8-b9da-8d54a6e67a94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:51.018998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-83bcb029-0627-4aa8-b9da-8d54a6e67a94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:51.032917 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bcb029-0627-4aa8-b9da-8d54a6e67a94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:51.034372 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bcb029-0627-4aa8-b9da-8d54a6e67a94 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:51.047935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1250/2512] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:33:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:33:54.907077 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36785e77-dfcb-4088-8ded-ef72942e72b5 service swift] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:54.910222 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-36785e77-dfcb-4088-8ded-ef72942e72b5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:54.935887 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36785e77-dfcb-4088-8ded-ef72942e72b5 service swift] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:54.941840 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c74b8f-2a91-4549-a6d4-ee72408da9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:54.943721 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c74b8f-2a91-4549-a6d4-ee72408da9ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:54.944013 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c74b8f-2a91-4549-a6d4-ee72408da9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:54.944288 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c74b8f-2a91-4549-a6d4-ee72408da9ed admin admin] PATH_INFO: `/v3/users/95bc48b6541c4388b45435036a1c4525` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:54.945973 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c74b8f-2a91-4549-a6d4-ee72408da9ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=95bc48b6541c4388b45435036a1c4525)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:54.948269 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c74b8f-2a91-4549-a6d4-ee72408da9ed admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:55.025665 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-15c74b8f-2a91-4549-a6d4-ee72408da9ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95bc48b6541c4388b45435036a1c4525'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:55.044373 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-15c74b8f-2a91-4549-a6d4-ee72408da9ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95bc48b6541c4388b45435036a1c4525'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:55.055424 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-15c74b8f-2a91-4549-a6d4-ee72408da9ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95bc48b6541c4388b45435036a1c4525'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:55.069628 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-15c74b8f-2a91-4549-a6d4-ee72408da9ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95bc48b6541c4388b45435036a1c4525'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:55.070346 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-15c74b8f-2a91-4549-a6d4-ee72408da9ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95bc48b6541c4388b45435036a1c4525'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:55.080377 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1263/2513] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:33:54 2022] DELETE /identity/v3/users/95bc48b6541c4388b45435036a1c4525 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:33:55.309155 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bcb029-0627-4aa8-b9da-8d54a6e67a94 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:55.328473 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bcb029-0627-4aa8-b9da-8d54a6e67a94 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:55.369012 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:33:55.397992 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e653af6-ae7d-41ce-98ba-1ed18a34b77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:55.398904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e653af6-ae7d-41ce-98ba-1ed18a34b77f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:55.399177 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e653af6-ae7d-41ce-98ba-1ed18a34b77f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:55.399451 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e653af6-ae7d-41ce-98ba-1ed18a34b77f admin admin] PATH_INFO: `/v3/projects/b310c05b03394f11b63f0a28f13d67b6` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:55.400696 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e653af6-ae7d-41ce-98ba-1ed18a34b77f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b310c05b03394f11b63f0a28f13d67b6)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:55.403038 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e653af6-ae7d-41ce-98ba-1ed18a34b77f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:55.460101 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0e653af6-ae7d-41ce-98ba-1ed18a34b77f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b310c05b03394f11b63f0a28f13d67b6', 'request_id': 'req-0e653af6-ae7d-41ce-98ba-1ed18a34b77f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:55.467005 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0e653af6-ae7d-41ce-98ba-1ed18a34b77f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b310c05b03394f11b63f0a28f13d67b6', 'request_id': 'req-0e653af6-ae7d-41ce-98ba-1ed18a34b77f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:55.474376 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0e653af6-ae7d-41ce-98ba-1ed18a34b77f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b310c05b03394f11b63f0a28f13d67b6', 'request_id': 'req-0e653af6-ae7d-41ce-98ba-1ed18a34b77f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:55.489454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0e653af6-ae7d-41ce-98ba-1ed18a34b77f admin admin] The token cache is being invalidate because project b310c05b03394f11b63f0a28f13d67b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:33:55.489789 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0e653af6-ae7d-41ce-98ba-1ed18a34b77f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:55.490410 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0e653af6-ae7d-41ce-98ba-1ed18a34b77f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:55.491911 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1251/2514] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:33:55 2022] DELETE /identity/v3/projects/b310c05b03394f11b63f0a28f13d67b6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:33:55.496873 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1179f6-4774-42c0-baf5-a98c72773353 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:55.497230 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1179f6-4774-42c0-baf5-a98c72773353 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:55.497613 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1179f6-4774-42c0-baf5-a98c72773353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:55.518635 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-5d1179f6-4774-42c0-baf5-a98c72773353 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:33:55.520376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-5d1179f6-4774-42c0-baf5-a98c72773353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:55.563491 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:33:55.592048 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1264/2515] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:33:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:33:55.600241 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667f96b-732c-4f07-bcb8-8be5d683507a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:55.600509 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667f96b-732c-4f07-bcb8-8be5d683507a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:55.601551 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667f96b-732c-4f07-bcb8-8be5d683507a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:55.624002 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-4667f96b-732c-4f07-bcb8-8be5d683507a None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:33:55.625663 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4667f96b-732c-4f07-bcb8-8be5d683507a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:55.634502 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1252/2516] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:33:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:33:55.638909 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1179f6-4774-42c0-baf5-a98c72773353 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:55.658450 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1179f6-4774-42c0-baf5-a98c72773353 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:55.677236 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc645d94-0919-4fa9-ba48-7ec34b3c3d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:55.677932 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc645d94-0919-4fa9-ba48-7ec34b3c3d4e admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:55.678236 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc645d94-0919-4fa9-ba48-7ec34b3c3d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:55.678483 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc645d94-0919-4fa9-ba48-7ec34b3c3d4e admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:55.679851 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc645d94-0919-4fa9-ba48-7ec34b3c3d4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:33:55.680183 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc645d94-0919-4fa9-ba48-7ec34b3c3d4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:55.681941 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc645d94-0919-4fa9-ba48-7ec34b3c3d4e admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:55.700028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1265/2517] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:33:55.705103 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4667f96b-732c-4f07-bcb8-8be5d683507a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:55.714792 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4667f96b-732c-4f07-bcb8-8be5d683507a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:55.723189 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcfa18d-ca78-4282-a38b-ec13dc0d8978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:55.724001 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcfa18d-ca78-4282-a38b-ec13dc0d8978 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:55.724279 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcfa18d-ca78-4282-a38b-ec13dc0d8978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:55.724549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcfa18d-ca78-4282-a38b-ec13dc0d8978 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:55.725196 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcfa18d-ca78-4282-a38b-ec13dc0d8978 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:55.727332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcfa18d-ca78-4282-a38b-ec13dc0d8978 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:55.767390 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1253/2518] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:33:55 2022] POST /identity/v3/projects => generated 361 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:33:55.771623 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc645d94-0919-4fa9-ba48-7ec34b3c3d4e admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:55.782259 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc645d94-0919-4fa9-ba48-7ec34b3c3d4e admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:55.858452 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f50a957-6685-4c3c-a66b-574e11d76c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:55.858941 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50a957-6685-4c3c-a66b-574e11d76c40 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:55.859243 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50a957-6685-4c3c-a66b-574e11d76c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:55.859488 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50a957-6685-4c3c-a66b-574e11d76c40 admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:55.860130 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f50a957-6685-4c3c-a66b-574e11d76c40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:55.862430 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f50a957-6685-4c3c-a66b-574e11d76c40 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:55.891926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1266/2519] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:33:55 2022] POST /identity/v3/users => generated 281 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:33:55.896843 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcfa18d-ca78-4282-a38b-ec13dc0d8978 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:55.908373 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcfa18d-ca78-4282-a38b-ec13dc0d8978 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:55.913003 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102dc837-9ca2-4e5f-b7e1-115c12ad2a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:55.913647 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102dc837-9ca2-4e5f-b7e1-115c12ad2a33 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:55.913927 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102dc837-9ca2-4e5f-b7e1-115c12ad2a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:55.914199 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102dc837-9ca2-4e5f-b7e1-115c12ad2a33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:55.914863 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102dc837-9ca2-4e5f-b7e1-115c12ad2a33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:55.916423 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102dc837-9ca2-4e5f-b7e1-115c12ad2a33 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:55.927050 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1254/2520] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:33:55.931460 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f50a957-6685-4c3c-a66b-574e11d76c40 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:55.944012 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f50a957-6685-4c3c-a66b-574e11d76c40 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:55.948754 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c621201-6bc9-40ec-87c0-1ec08423716c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:55.949460 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c621201-6bc9-40ec-87c0-1ec08423716c admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:55.949799 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c621201-6bc9-40ec-87c0-1ec08423716c admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:55.950160 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c621201-6bc9-40ec-87c0-1ec08423716c admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:55.950863 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c621201-6bc9-40ec-87c0-1ec08423716c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:55.952603 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c621201-6bc9-40ec-87c0-1ec08423716c admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:55.963698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1267/2521] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:33:55.968543 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102dc837-9ca2-4e5f-b7e1-115c12ad2a33 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:55.979006 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102dc837-9ca2-4e5f-b7e1-115c12ad2a33 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:55.984475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c57dd-351c-43cb-9414-e8eff292c544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:55.985408 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c57dd-351c-43cb-9414-e8eff292c544 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:55.985780 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c57dd-351c-43cb-9414-e8eff292c544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:55.986399 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c57dd-351c-43cb-9414-e8eff292c544 admin admin] PATH_INFO: `/v3/projects/dc51bbaabb9e475f878ab7409099e979/users/021e980e8bfe4399b5f1cceddbfa8542/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:56.004532 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c57dd-351c-43cb-9414-e8eff292c544 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc51bbaabb9e475f878ab7409099e979, user_id=021e980e8bfe4399b5f1cceddbfa8542, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:56.007229 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c57dd-351c-43cb-9414-e8eff292c544 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:56.022349 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1255/2522] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:33:55 2022] PUT /identity/v3/projects/dc51bbaabb9e475f878ab7409099e979/users/021e980e8bfe4399b5f1cceddbfa8542/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:33:56.035026 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51af44-73cf-49b0-993e-fbe4bc508cd4 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:56.035574 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51af44-73cf-49b0-993e-fbe4bc508cd4 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:56.035914 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51af44-73cf-49b0-993e-fbe4bc508cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:56.066462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-3d51af44-73cf-49b0-993e-fbe4bc508cd4 None None] MFA Rules not processed for user `021e980e8bfe4399b5f1cceddbfa8542`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:33:56.067822 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-3d51af44-73cf-49b0-993e-fbe4bc508cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:56.145835 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1268/2523] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:33:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:33:56.150217 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c57dd-351c-43cb-9414-e8eff292c544 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:56.223188 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c57dd-351c-43cb-9414-e8eff292c544 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:56.228042 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306b9e1-d39d-454c-8d88-6cd6ef2a5417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:56.228652 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306b9e1-d39d-454c-8d88-6cd6ef2a5417 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:56.228947 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306b9e1-d39d-454c-8d88-6cd6ef2a5417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:56.229197 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306b9e1-d39d-454c-8d88-6cd6ef2a5417 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:56.229903 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a306b9e1-d39d-454c-8d88-6cd6ef2a5417 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:56.231884 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a306b9e1-d39d-454c-8d88-6cd6ef2a5417 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:56.238369 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:33:56.263854 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1256/2524] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:33:56 2022] POST /identity/v3/projects => generated 363 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:33:56.268690 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51af44-73cf-49b0-993e-fbe4bc508cd4 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:56.279547 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51af44-73cf-49b0-993e-fbe4bc508cd4 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:56.321185 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:33:56.353528 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7aecb-edb1-4800-8d7c-26d305e022ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:56.354291 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7aecb-edb1-4800-8d7c-26d305e022ad admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:56.354619 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7aecb-edb1-4800-8d7c-26d305e022ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:56.354963 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7aecb-edb1-4800-8d7c-26d305e022ad admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:56.355608 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7aecb-edb1-4800-8d7c-26d305e022ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:56.357554 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7aecb-edb1-4800-8d7c-26d305e022ad admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:56.380181 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1269/2525] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:33:56 2022] POST /identity/v3/users => generated 288 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:33:56.384378 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306b9e1-d39d-454c-8d88-6cd6ef2a5417 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:56.397299 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306b9e1-d39d-454c-8d88-6cd6ef2a5417 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:56.401966 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a4178-b246-431c-9de4-94d15012bc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:56.402573 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a4178-b246-431c-9de4-94d15012bc8b admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:56.402852 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a4178-b246-431c-9de4-94d15012bc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:56.403124 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a4178-b246-431c-9de4-94d15012bc8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:56.403761 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a4178-b246-431c-9de4-94d15012bc8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:56.405966 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a4178-b246-431c-9de4-94d15012bc8b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:56.417288 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1257/2526] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:33:56.421344 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7aecb-edb1-4800-8d7c-26d305e022ad admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:56.433604 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7aecb-edb1-4800-8d7c-26d305e022ad admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:56.440144 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b260aaa5-bd8d-4993-b046-3f857a3fd254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:56.441365 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b260aaa5-bd8d-4993-b046-3f857a3fd254 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:56.441809 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b260aaa5-bd8d-4993-b046-3f857a3fd254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:56.442326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b260aaa5-bd8d-4993-b046-3f857a3fd254 admin admin] PATH_INFO: `/v3/projects/39bd1317ea554cbe814bd0e004585863/users/131990dd69aa4f51972fe5b2ba8eef23/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:56.463437 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b260aaa5-bd8d-4993-b046-3f857a3fd254 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39bd1317ea554cbe814bd0e004585863, user_id=131990dd69aa4f51972fe5b2ba8eef23, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:56.466253 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b260aaa5-bd8d-4993-b046-3f857a3fd254 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:56.481668 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1270/2527] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:33:56 2022] PUT /identity/v3/projects/39bd1317ea554cbe814bd0e004585863/users/131990dd69aa4f51972fe5b2ba8eef23/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:33:56.485921 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a4178-b246-431c-9de4-94d15012bc8b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:56.497276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a4178-b246-431c-9de4-94d15012bc8b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:56.558184 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1f983-460f-4a8e-b537-d691bdc6993d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:56.562034 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1f983-460f-4a8e-b537-d691bdc6993d admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:56.562360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1f983-460f-4a8e-b537-d691bdc6993d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:56.562638 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1f983-460f-4a8e-b537-d691bdc6993d admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:56.563330 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c1f983-460f-4a8e-b537-d691bdc6993d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:56.564954 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c1f983-460f-4a8e-b537-d691bdc6993d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:56.575291 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1258/2528] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:33:56.580113 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b260aaa5-bd8d-4993-b046-3f857a3fd254 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:56.595411 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b260aaa5-bd8d-4993-b046-3f857a3fd254 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:56.600157 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c9ec41-e162-4e80-87a5-54d3c87c8519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:56.601065 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9ec41-e162-4e80-87a5-54d3c87c8519 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:56.601399 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9ec41-e162-4e80-87a5-54d3c87c8519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:56.601697 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9ec41-e162-4e80-87a5-54d3c87c8519 admin admin] PATH_INFO: `/v3/projects/39bd1317ea554cbe814bd0e004585863/users/131990dd69aa4f51972fe5b2ba8eef23/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:56.603610 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c9ec41-e162-4e80-87a5-54d3c87c8519 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39bd1317ea554cbe814bd0e004585863, user_id=131990dd69aa4f51972fe5b2ba8eef23, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:56.606280 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c9ec41-e162-4e80-87a5-54d3c87c8519 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:56.622173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1271/2529] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:33:56 2022] PUT /identity/v3/projects/39bd1317ea554cbe814bd0e004585863/users/131990dd69aa4f51972fe5b2ba8eef23/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:33:56.633206 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9dc87f-8990-4400-b9e1-6f680dd5a7f9 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:56.633597 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9dc87f-8990-4400-b9e1-6f680dd5a7f9 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:56.633941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9dc87f-8990-4400-b9e1-6f680dd5a7f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:56.663906 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-6a9dc87f-8990-4400-b9e1-6f680dd5a7f9 None None] MFA Rules not processed for user `131990dd69aa4f51972fe5b2ba8eef23`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:33:56.665042 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6a9dc87f-8990-4400-b9e1-6f680dd5a7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:56.740301 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1259/2530] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:33:56 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:33:56.753092 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c9ec41-e162-4e80-87a5-54d3c87c8519 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:56.758029 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-98c9ec41-e162-4e80-87a5-54d3c87c8519 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:56.825374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c9ec41-e162-4e80-87a5-54d3c87c8519 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:56.828900 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e6150-b5b4-4204-b83b-783c97ba04d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:56.829566 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e6150-b5b4-4204-b83b-783c97ba04d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:56.829847 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e6150-b5b4-4204-b83b-783c97ba04d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:56.830184 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e6150-b5b4-4204-b83b-783c97ba04d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:56.840297 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337e6150-b5b4-4204-b83b-783c97ba04d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:56.842112 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337e6150-b5b4-4204-b83b-783c97ba04d5 service cinder] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:56.861765 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1272/2531] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:33:56 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:33:57.366923 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9dc87f-8990-4400-b9e1-6f680dd5a7f9 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:57.437228 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:33:57.442001 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9dc87f-8990-4400-b9e1-6f680dd5a7f9 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:57.448101 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcea08-0f7f-4595-892a-0b78a267e0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:57.448842 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcea08-0f7f-4595-892a-0b78a267e0a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:57.452901 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcea08-0f7f-4595-892a-0b78a267e0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:57.452901 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcea08-0f7f-4595-892a-0b78a267e0a1 admin admin] PATH_INFO: `/v3/users/021e980e8bfe4399b5f1cceddbfa8542` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:57.452901 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dcea08-0f7f-4595-892a-0b78a267e0a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=021e980e8bfe4399b5f1cceddbfa8542)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:57.452901 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dcea08-0f7f-4595-892a-0b78a267e0a1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:57.509230 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-57dcea08-0f7f-4595-892a-0b78a267e0a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '021e980e8bfe4399b5f1cceddbfa8542'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:57.516982 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-57dcea08-0f7f-4595-892a-0b78a267e0a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '021e980e8bfe4399b5f1cceddbfa8542'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:57.531216 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-57dcea08-0f7f-4595-892a-0b78a267e0a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '021e980e8bfe4399b5f1cceddbfa8542'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:57.540173 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-57dcea08-0f7f-4595-892a-0b78a267e0a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '021e980e8bfe4399b5f1cceddbfa8542'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:57.540901 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-57dcea08-0f7f-4595-892a-0b78a267e0a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '021e980e8bfe4399b5f1cceddbfa8542'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:57.549130 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1260/2532] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:33:57 2022] DELETE /identity/v3/users/021e980e8bfe4399b5f1cceddbfa8542 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:33:57.554148 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e7949-3c26-4141-8e51-379e3c29fd20 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:57.554663 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e7949-3c26-4141-8e51-379e3c29fd20 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:57.554936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e7949-3c26-4141-8e51-379e3c29fd20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:57.574781 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-d34e7949-3c26-4141-8e51-379e3c29fd20 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:33:57.575977 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-d34e7949-3c26-4141-8e51-379e3c29fd20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:57.646191 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1273/2533] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:33:57.661636 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcea08-0f7f-4595-892a-0b78a267e0a1 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:57.663345 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-57dcea08-0f7f-4595-892a-0b78a267e0a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:57.732889 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcea08-0f7f-4595-892a-0b78a267e0a1 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:57.736967 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b93540-3ef0-4b9f-91a5-096cf6e104be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:57.737613 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b93540-3ef0-4b9f-91a5-096cf6e104be service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:57.737893 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b93540-3ef0-4b9f-91a5-096cf6e104be service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:57.738167 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b93540-3ef0-4b9f-91a5-096cf6e104be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:57.748637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b93540-3ef0-4b9f-91a5-096cf6e104be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:57.750377 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b93540-3ef0-4b9f-91a5-096cf6e104be service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:57.760376 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1261/2534] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:33:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:33:58.058639 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34e7949-3c26-4141-8e51-379e3c29fd20 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:58.060095 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-d34e7949-3c26-4141-8e51-379e3c29fd20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:58.075054 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34e7949-3c26-4141-8e51-379e3c29fd20 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:58.080976 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a55ad5-da04-44d3-9f90-0a017707dd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:58.081907 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a55ad5-da04-44d3-9f90-0a017707dd31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:58.082353 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a55ad5-da04-44d3-9f90-0a017707dd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:58.082928 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a55ad5-da04-44d3-9f90-0a017707dd31 admin admin] PATH_INFO: `/v3/projects/dc51bbaabb9e475f878ab7409099e979` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:58.084618 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a55ad5-da04-44d3-9f90-0a017707dd31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc51bbaabb9e475f878ab7409099e979)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:58.086892 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a55ad5-da04-44d3-9f90-0a017707dd31 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:58.134662 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:33:58.153239 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-15a55ad5-da04-44d3-9f90-0a017707dd31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc51bbaabb9e475f878ab7409099e979', 'request_id': 'req-15a55ad5-da04-44d3-9f90-0a017707dd31'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:58.163674 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-15a55ad5-da04-44d3-9f90-0a017707dd31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc51bbaabb9e475f878ab7409099e979', 'request_id': 'req-15a55ad5-da04-44d3-9f90-0a017707dd31'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:58.170086 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-15a55ad5-da04-44d3-9f90-0a017707dd31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc51bbaabb9e475f878ab7409099e979', 'request_id': 'req-15a55ad5-da04-44d3-9f90-0a017707dd31'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:58.182582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-15a55ad5-da04-44d3-9f90-0a017707dd31 admin admin] The token cache is being invalidate because project dc51bbaabb9e475f878ab7409099e979 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:33:58.182869 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-15a55ad5-da04-44d3-9f90-0a017707dd31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:58.183535 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-15a55ad5-da04-44d3-9f90-0a017707dd31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:58.185036 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1274/2535] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:33:58 2022] DELETE /identity/v3/projects/dc51bbaabb9e475f878ab7409099e979 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:33:58.189033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b93540-3ef0-4b9f-91a5-096cf6e104be service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:58.199070 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b93540-3ef0-4b9f-91a5-096cf6e104be service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:58.258084 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062877f3-66e9-45b5-84d9-f6a38365a752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:58.258645 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062877f3-66e9-45b5-84d9-f6a38365a752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:58.258927 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062877f3-66e9-45b5-84d9-f6a38365a752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:58.259196 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062877f3-66e9-45b5-84d9-f6a38365a752 admin admin] PATH_INFO: `/v3/users/131990dd69aa4f51972fe5b2ba8eef23` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:58.260316 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062877f3-66e9-45b5-84d9-f6a38365a752 admin admin] RBAC: Authorizing `identity:delete_user(user_id=131990dd69aa4f51972fe5b2ba8eef23)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:58.262271 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062877f3-66e9-45b5-84d9-f6a38365a752 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:58.283573 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:33:58.309586 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-062877f3-66e9-45b5-84d9-f6a38365a752 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '131990dd69aa4f51972fe5b2ba8eef23'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:58.316524 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-062877f3-66e9-45b5-84d9-f6a38365a752 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '131990dd69aa4f51972fe5b2ba8eef23'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:58.328791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-062877f3-66e9-45b5-84d9-f6a38365a752 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '131990dd69aa4f51972fe5b2ba8eef23'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:58.336383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-062877f3-66e9-45b5-84d9-f6a38365a752 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '131990dd69aa4f51972fe5b2ba8eef23'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:58.336999 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-062877f3-66e9-45b5-84d9-f6a38365a752 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '131990dd69aa4f51972fe5b2ba8eef23'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:58.342811 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1262/2536] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:33:58 2022] DELETE /identity/v3/users/131990dd69aa4f51972fe5b2ba8eef23 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:33:58.621274 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a55ad5-da04-44d3-9f90-0a017707dd31 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:58.623619 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-15a55ad5-da04-44d3-9f90-0a017707dd31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:58.639410 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a55ad5-da04-44d3-9f90-0a017707dd31 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:58.697962 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6cf817-ec14-45e9-8061-be9f6e47324c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:58.698590 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6cf817-ec14-45e9-8061-be9f6e47324c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:58.698866 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6cf817-ec14-45e9-8061-be9f6e47324c admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:58.699134 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6cf817-ec14-45e9-8061-be9f6e47324c admin admin] PATH_INFO: `/v3/projects/39bd1317ea554cbe814bd0e004585863` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:58.700217 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6cf817-ec14-45e9-8061-be9f6e47324c admin admin] RBAC: Authorizing `identity:delete_project(project_id=39bd1317ea554cbe814bd0e004585863)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:58.702692 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6cf817-ec14-45e9-8061-be9f6e47324c admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:58.750830 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-ee6cf817-ec14-45e9-8061-be9f6e47324c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39bd1317ea554cbe814bd0e004585863', 'request_id': 'req-ee6cf817-ec14-45e9-8061-be9f6e47324c'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:58.757579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-ee6cf817-ec14-45e9-8061-be9f6e47324c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39bd1317ea554cbe814bd0e004585863', 'request_id': 'req-ee6cf817-ec14-45e9-8061-be9f6e47324c'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:58.763132 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-ee6cf817-ec14-45e9-8061-be9f6e47324c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39bd1317ea554cbe814bd0e004585863', 'request_id': 'req-ee6cf817-ec14-45e9-8061-be9f6e47324c'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:58.774766 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-ee6cf817-ec14-45e9-8061-be9f6e47324c admin admin] The token cache is being invalidate because project 39bd1317ea554cbe814bd0e004585863 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:33:58.775055 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-ee6cf817-ec14-45e9-8061-be9f6e47324c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:58.775682 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-ee6cf817-ec14-45e9-8061-be9f6e47324c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:33:58.777150 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1275/2537] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:33:58 2022] DELETE /identity/v3/projects/39bd1317ea554cbe814bd0e004585863 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:33:58.782063 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d3860-1394-48fd-9b4d-64e71c1f5f8a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:58.782546 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d3860-1394-48fd-9b4d-64e71c1f5f8a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:58.782648 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d3860-1394-48fd-9b4d-64e71c1f5f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:58.801809 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-bb7d3860-1394-48fd-9b4d-64e71c1f5f8a None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:33:58.802930 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-bb7d3860-1394-48fd-9b4d-64e71c1f5f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:58.864988 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1263/2538] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:33:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:33:58.872824 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c70de2-feb1-42f4-b0cd-a5f16d5058dc None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:58.873133 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c70de2-feb1-42f4-b0cd-a5f16d5058dc None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:58.873479 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c70de2-feb1-42f4-b0cd-a5f16d5058dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:58.893693 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-e1c70de2-feb1-42f4-b0cd-a5f16d5058dc None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:33:58.894871 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-e1c70de2-feb1-42f4-b0cd-a5f16d5058dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:58.902614 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1276/2539] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:33:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:33:58.906837 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d3860-1394-48fd-9b4d-64e71c1f5f8a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:58.916383 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d3860-1394-48fd-9b4d-64e71c1f5f8a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:58.920916 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d3471-b613-497f-9c6f-bf911637911f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:58.921537 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d3471-b613-497f-9c6f-bf911637911f admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:58.921819 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d3471-b613-497f-9c6f-bf911637911f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:58.922094 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d3471-b613-497f-9c6f-bf911637911f admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:58.922732 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d3471-b613-497f-9c6f-bf911637911f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:33:58.923033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d3471-b613-497f-9c6f-bf911637911f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:58.924607 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d3471-b613-497f-9c6f-bf911637911f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:58.935805 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1264/2540] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:33:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:33:58.940464 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c70de2-feb1-42f4-b0cd-a5f16d5058dc None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:58.949493 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c70de2-feb1-42f4-b0cd-a5f16d5058dc None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:58.954776 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80819a62-2503-4525-9ae6-020055ce64ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:58.955375 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80819a62-2503-4525-9ae6-020055ce64ed admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:58.955636 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80819a62-2503-4525-9ae6-020055ce64ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:58.955905 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80819a62-2503-4525-9ae6-020055ce64ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:58.956668 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80819a62-2503-4525-9ae6-020055ce64ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:58.958564 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80819a62-2503-4525-9ae6-020055ce64ed admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:58.984550 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1277/2541] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:33:58 2022] POST /identity/v3/projects => generated 379 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:33:58.989032 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d3471-b613-497f-9c6f-bf911637911f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:58.998737 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d3471-b613-497f-9c6f-bf911637911f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:59.055123 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e06494-ca61-4f07-8e44-37a59997c656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:59.055667 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e06494-ca61-4f07-8e44-37a59997c656 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:59.055951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e06494-ca61-4f07-8e44-37a59997c656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:59.056222 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e06494-ca61-4f07-8e44-37a59997c656 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:59.056841 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e06494-ca61-4f07-8e44-37a59997c656 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:59.058766 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e06494-ca61-4f07-8e44-37a59997c656 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:59.080000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1265/2542] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:33:58 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:33:59.084151 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80819a62-2503-4525-9ae6-020055ce64ed admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:59.094376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80819a62-2503-4525-9ae6-020055ce64ed admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:59.098875 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dbbe68-a8b0-45c4-b440-737dec9aabae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:59.099462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dbbe68-a8b0-45c4-b440-737dec9aabae admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:59.099740 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dbbe68-a8b0-45c4-b440-737dec9aabae admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:59.100010 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dbbe68-a8b0-45c4-b440-737dec9aabae admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:59.100638 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dbbe68-a8b0-45c4-b440-737dec9aabae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:59.102246 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dbbe68-a8b0-45c4-b440-737dec9aabae admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:59.112660 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1278/2543] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:33:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:33:59.116454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e06494-ca61-4f07-8e44-37a59997c656 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:59.126270 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e06494-ca61-4f07-8e44-37a59997c656 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:59.130906 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7a4824-645e-49d6-ab1c-caa634310016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:59.131493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a4824-645e-49d6-ab1c-caa634310016 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:59.131774 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a4824-645e-49d6-ab1c-caa634310016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:59.132047 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a4824-645e-49d6-ab1c-caa634310016 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:59.132680 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7a4824-645e-49d6-ab1c-caa634310016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:59.134281 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7a4824-645e-49d6-ab1c-caa634310016 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:59.144401 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1266/2544] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:33:59 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:33:59.148195 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dbbe68-a8b0-45c4-b440-737dec9aabae admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:59.163586 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dbbe68-a8b0-45c4-b440-737dec9aabae admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:59.165609 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf97819-0676-4d6e-b020-83fb3f18c501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:59.166261 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf97819-0676-4d6e-b020-83fb3f18c501 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:59.166540 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf97819-0676-4d6e-b020-83fb3f18c501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:59.166819 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf97819-0676-4d6e-b020-83fb3f18c501 admin admin] PATH_INFO: `/v3/projects/cc7667a772e6467d8efd6abeb42462c6/users/97b85a992b8a4a779021d9c2a1542c5b/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:59.169940 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:33:59.187606 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf97819-0676-4d6e-b020-83fb3f18c501 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc7667a772e6467d8efd6abeb42462c6, user_id=97b85a992b8a4a779021d9c2a1542c5b, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:59.190519 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf97819-0676-4d6e-b020-83fb3f18c501 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:59.205472 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1279/2545] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:33:59 2022] PUT /identity/v3/projects/cc7667a772e6467d8efd6abeb42462c6/users/97b85a992b8a4a779021d9c2a1542c5b/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:33:59.216891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93e35b-21f0-45fe-b87d-8d02d4e4037f None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:59.217182 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93e35b-21f0-45fe-b87d-8d02d4e4037f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:59.217567 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93e35b-21f0-45fe-b87d-8d02d4e4037f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:59.226291 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:33:59.249584 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-3a93e35b-21f0-45fe-b87d-8d02d4e4037f None None] MFA Rules not processed for user `97b85a992b8a4a779021d9c2a1542c5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:33:59.250828 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3a93e35b-21f0-45fe-b87d-8d02d4e4037f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:59.316447 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1267/2546] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:33:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:33:59.320839 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf97819-0676-4d6e-b020-83fb3f18c501 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:59.387187 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf97819-0676-4d6e-b020-83fb3f18c501 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:59.391882 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e980f26-154c-4849-bad0-ef228b7473eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:59.392489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e980f26-154c-4849-bad0-ef228b7473eb admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:59.392769 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e980f26-154c-4849-bad0-ef228b7473eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:59.393042 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e980f26-154c-4849-bad0-ef228b7473eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:59.393695 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e980f26-154c-4849-bad0-ef228b7473eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:59.395622 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e980f26-154c-4849-bad0-ef228b7473eb admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:59.423909 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1280/2547] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:33:59 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:33:59.427928 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a93e35b-21f0-45fe-b87d-8d02d4e4037f None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:59.439181 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a93e35b-21f0-45fe-b87d-8d02d4e4037f None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:59.497430 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc9333-259c-4924-bf83-c8e0e667de97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:59.498036 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc9333-259c-4924-bf83-c8e0e667de97 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:59.498316 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc9333-259c-4924-bf83-c8e0e667de97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:59.498590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc9333-259c-4924-bf83-c8e0e667de97 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:59.499224 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bc9333-259c-4924-bf83-c8e0e667de97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:59.501113 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bc9333-259c-4924-bf83-c8e0e667de97 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:59.521254 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1268/2548] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:33:59 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:33:59.525188 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e980f26-154c-4849-bad0-ef228b7473eb admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:59.535757 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e980f26-154c-4849-bad0-ef228b7473eb admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:59.540401 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf6f9e-9348-4027-be52-f98b45fc48eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:59.540994 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf6f9e-9348-4027-be52-f98b45fc48eb admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:59.541302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf6f9e-9348-4027-be52-f98b45fc48eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:59.541588 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf6f9e-9348-4027-be52-f98b45fc48eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:59.542224 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccf6f9e-9348-4027-be52-f98b45fc48eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:59.543803 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccf6f9e-9348-4027-be52-f98b45fc48eb admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:59.554705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1281/2549] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:33:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:33:59.558418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc9333-259c-4924-bf83-c8e0e667de97 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:59.568116 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc9333-259c-4924-bf83-c8e0e667de97 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:59.572633 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d58d6-8c44-4e4d-bd48-beaaa4021d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:59.573215 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d58d6-8c44-4e4d-bd48-beaaa4021d66 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:59.573525 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d58d6-8c44-4e4d-bd48-beaaa4021d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:59.573790 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d58d6-8c44-4e4d-bd48-beaaa4021d66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:59.574415 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6d58d6-8c44-4e4d-bd48-beaaa4021d66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:59.576001 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6d58d6-8c44-4e4d-bd48-beaaa4021d66 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:59.585590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1269/2550] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:33:59 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:33:59.589399 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf6f9e-9348-4027-be52-f98b45fc48eb admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:59.599511 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf6f9e-9348-4027-be52-f98b45fc48eb admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:59.604112 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970aea6-98d8-42f3-9ef1-8c11ae89fbe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:59.604744 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970aea6-98d8-42f3-9ef1-8c11ae89fbe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:59.605023 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970aea6-98d8-42f3-9ef1-8c11ae89fbe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:59.605326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970aea6-98d8-42f3-9ef1-8c11ae89fbe2 admin admin] PATH_INFO: `/v3/projects/b44fee2589724085a5347698fb9ddab8/users/a83a21d5a6e64f2fbc381396b5cebfb7/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:59.620169 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c970aea6-98d8-42f3-9ef1-8c11ae89fbe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b44fee2589724085a5347698fb9ddab8, user_id=a83a21d5a6e64f2fbc381396b5cebfb7, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:59.622824 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c970aea6-98d8-42f3-9ef1-8c11ae89fbe2 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:59.635165 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1282/2551] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:33:59 2022] PUT /identity/v3/projects/b44fee2589724085a5347698fb9ddab8/users/a83a21d5a6e64f2fbc381396b5cebfb7/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:33:59.645594 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590aa2a-0e4e-48c3-bcec-1c16dd02bcfc None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:59.645904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590aa2a-0e4e-48c3-bcec-1c16dd02bcfc None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:59.646179 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590aa2a-0e4e-48c3-bcec-1c16dd02bcfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:59.675165 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-7590aa2a-0e4e-48c3-bcec-1c16dd02bcfc None None] MFA Rules not processed for user `a83a21d5a6e64f2fbc381396b5cebfb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:33:59.676369 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7590aa2a-0e4e-48c3-bcec-1c16dd02bcfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:33:59.738242 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1270/2552] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:33:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:33:59.742785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970aea6-98d8-42f3-9ef1-8c11ae89fbe2 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:59.804102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970aea6-98d8-42f3-9ef1-8c11ae89fbe2 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:59.808754 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5495fdc-16ed-46bc-b0ac-adba9b71e5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:59.809398 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5495fdc-16ed-46bc-b0ac-adba9b71e5a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:59.809673 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5495fdc-16ed-46bc-b0ac-adba9b71e5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:59.809939 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5495fdc-16ed-46bc-b0ac-adba9b71e5a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:59.810545 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5495fdc-16ed-46bc-b0ac-adba9b71e5a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:59.812989 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5495fdc-16ed-46bc-b0ac-adba9b71e5a1 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:59.839284 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1283/2553] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:33:59 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:33:59.843206 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590aa2a-0e4e-48c3-bcec-1c16dd02bcfc None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:59.848619 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:33:59.855309 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590aa2a-0e4e-48c3-bcec-1c16dd02bcfc None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:59.911280 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ff3ce-bb14-4e5f-819b-8181a18683bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:59.911886 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ff3ce-bb14-4e5f-819b-8181a18683bd admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:59.912195 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ff3ce-bb14-4e5f-819b-8181a18683bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:59.912895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ff3ce-bb14-4e5f-819b-8181a18683bd admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:59.913097 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ff3ce-bb14-4e5f-819b-8181a18683bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:59.915095 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ff3ce-bb14-4e5f-819b-8181a18683bd admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:59.937109 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1271/2554] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:33:59 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:33:59.941466 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5495fdc-16ed-46bc-b0ac-adba9b71e5a1 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:59.951472 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5495fdc-16ed-46bc-b0ac-adba9b71e5a1 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:59.955961 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba5fae-63c4-48e6-91f0-97c1fb9c534b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:59.956548 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba5fae-63c4-48e6-91f0-97c1fb9c534b admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:59.956821 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba5fae-63c4-48e6-91f0-97c1fb9c534b admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:59.957088 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba5fae-63c4-48e6-91f0-97c1fb9c534b admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:33:59.957749 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aba5fae-63c4-48e6-91f0-97c1fb9c534b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:33:59.959316 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aba5fae-63c4-48e6-91f0-97c1fb9c534b admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:33:59.969526 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1284/2555] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:33:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:33:59.973385 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ff3ce-bb14-4e5f-819b-8181a18683bd admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:33:59.983769 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ff3ce-bb14-4e5f-819b-8181a18683bd admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:33:59.988624 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f6ccf-a193-4da4-baeb-e82322604077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:33:59.989280 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f6ccf-a193-4da4-baeb-e82322604077 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:33:59.989564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f6ccf-a193-4da4-baeb-e82322604077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:33:59.989837 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f6ccf-a193-4da4-baeb-e82322604077 admin admin] PATH_INFO: `/v3/projects/beeebcf3b7324139b11fef89149b38e0/users/d78e923c2181497d8fa13447ce2836d7/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:00.005097 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23f6ccf-a193-4da4-baeb-e82322604077 admin admin] RBAC: Authorizing `identity:create_grant(project_id=beeebcf3b7324139b11fef89149b38e0, user_id=d78e923c2181497d8fa13447ce2836d7, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:00.007772 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23f6ccf-a193-4da4-baeb-e82322604077 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:00.020631 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1272/2556] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:33:59 2022] PUT /identity/v3/projects/beeebcf3b7324139b11fef89149b38e0/users/d78e923c2181497d8fa13447ce2836d7/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:34:00.024701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba5fae-63c4-48e6-91f0-97c1fb9c534b admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:00.034721 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba5fae-63c4-48e6-91f0-97c1fb9c534b admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:00.038371 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:34:00.092534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776a520-1ab4-4d23-87e5-38a2a109eae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:00.093065 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776a520-1ab4-4d23-87e5-38a2a109eae5 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:00.093387 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776a520-1ab4-4d23-87e5-38a2a109eae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:00.093664 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776a520-1ab4-4d23-87e5-38a2a109eae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:00.094302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0776a520-1ab4-4d23-87e5-38a2a109eae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:00.095893 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0776a520-1ab4-4d23-87e5-38a2a109eae5 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:00.105620 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1285/2557] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:34:00.110194 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f6ccf-a193-4da4-baeb-e82322604077 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:00.122278 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f6ccf-a193-4da4-baeb-e82322604077 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:00.127575 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43db01-aae3-4b26-b7ad-69b43aa005e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:00.128233 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43db01-aae3-4b26-b7ad-69b43aa005e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:00.128512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43db01-aae3-4b26-b7ad-69b43aa005e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:00.128784 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43db01-aae3-4b26-b7ad-69b43aa005e1 admin admin] PATH_INFO: `/v3/projects/beeebcf3b7324139b11fef89149b38e0/users/d78e923c2181497d8fa13447ce2836d7/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:00.130806 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43db01-aae3-4b26-b7ad-69b43aa005e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=beeebcf3b7324139b11fef89149b38e0, user_id=d78e923c2181497d8fa13447ce2836d7, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:00.133453 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43db01-aae3-4b26-b7ad-69b43aa005e1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:00.143878 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1273/2558] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:34:00 2022] PUT /identity/v3/projects/beeebcf3b7324139b11fef89149b38e0/users/d78e923c2181497d8fa13447ce2836d7/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:34:00.155258 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25748e4f-539f-4d86-b986-120726eac0fe None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:00.155545 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25748e4f-539f-4d86-b986-120726eac0fe None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:00.155818 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25748e4f-539f-4d86-b986-120726eac0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:00.184089 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-25748e4f-539f-4d86-b986-120726eac0fe None None] MFA Rules not processed for user `d78e923c2181497d8fa13447ce2836d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:34:00.185364 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-25748e4f-539f-4d86-b986-120726eac0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:00.254955 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1286/2559] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:34:00 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:34:00.266410 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43db01-aae3-4b26-b7ad-69b43aa005e1 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:00.268133 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ea43db01-aae3-4b26-b7ad-69b43aa005e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:00.328945 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43db01-aae3-4b26-b7ad-69b43aa005e1 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:00.332704 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ca6af-3112-4672-b43b-ce6d38423d9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:00.333346 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ca6af-3112-4672-b43b-ce6d38423d9a service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:00.333630 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ca6af-3112-4672-b43b-ce6d38423d9a service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:00.333904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ca6af-3112-4672-b43b-ce6d38423d9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:00.343796 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ca6af-3112-4672-b43b-ce6d38423d9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:00.345171 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ca6af-3112-4672-b43b-ce6d38423d9a service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:00.355263 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1274/2560] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:34:00 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 20:34:00.397690 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25748e4f-539f-4d86-b986-120726eac0fe None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:00.399077 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-25748e4f-539f-4d86-b986-120726eac0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:00.411445 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25748e4f-539f-4d86-b986-120726eac0fe None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:00.415256 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453870c6-9f92-4fef-b22c-7a1873b70680 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:00.415855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453870c6-9f92-4fef-b22c-7a1873b70680 service cinder] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:00.416155 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453870c6-9f92-4fef-b22c-7a1873b70680 service cinder] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:00.416454 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453870c6-9f92-4fef-b22c-7a1873b70680 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:00.474328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453870c6-9f92-4fef-b22c-7a1873b70680 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:00.475779 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453870c6-9f92-4fef-b22c-7a1873b70680 service cinder] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:00.485544 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1287/2561] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:34:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:34:02.279087 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ca6af-3112-4672-b43b-ce6d38423d9a service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:02.281365 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-515ca6af-3112-4672-b43b-ce6d38423d9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:02.338897 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:34:02.362530 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ca6af-3112-4672-b43b-ce6d38423d9a service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:02.368346 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e094d68-b1e5-46af-9c87-aab221f7ba28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:02.368944 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e094d68-b1e5-46af-9c87-aab221f7ba28 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:02.369221 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e094d68-b1e5-46af-9c87-aab221f7ba28 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:02.369637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e094d68-b1e5-46af-9c87-aab221f7ba28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:02.371484 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8e094d68-b1e5-46af-9c87-aab221f7ba28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:02.384712 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e094d68-b1e5-46af-9c87-aab221f7ba28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:02.386471 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e094d68-b1e5-46af-9c87-aab221f7ba28 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:02.397615 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1275/2562] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:34:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:34:03.391829 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453870c6-9f92-4fef-b22c-7a1873b70680 service cinder] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:03.422218 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:34:03.454691 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453870c6-9f92-4fef-b22c-7a1873b70680 service cinder] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:03.459302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc65b81-f1f5-4a7c-8f33-34b2f459fd02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:03.459920 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc65b81-f1f5-4a7c-8f33-34b2f459fd02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:03.460194 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc65b81-f1f5-4a7c-8f33-34b2f459fd02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:03.460461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc65b81-f1f5-4a7c-8f33-34b2f459fd02 admin admin] PATH_INFO: `/v3/users/97b85a992b8a4a779021d9c2a1542c5b` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:03.461574 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc65b81-f1f5-4a7c-8f33-34b2f459fd02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97b85a992b8a4a779021d9c2a1542c5b)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:03.463473 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc65b81-f1f5-4a7c-8f33-34b2f459fd02 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:03.508488 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6bc65b81-f1f5-4a7c-8f33-34b2f459fd02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97b85a992b8a4a779021d9c2a1542c5b'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:03.520167 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6bc65b81-f1f5-4a7c-8f33-34b2f459fd02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97b85a992b8a4a779021d9c2a1542c5b'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:03.526807 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6bc65b81-f1f5-4a7c-8f33-34b2f459fd02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97b85a992b8a4a779021d9c2a1542c5b'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:03.534775 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6bc65b81-f1f5-4a7c-8f33-34b2f459fd02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97b85a992b8a4a779021d9c2a1542c5b'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:03.535395 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6bc65b81-f1f5-4a7c-8f33-34b2f459fd02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97b85a992b8a4a779021d9c2a1542c5b'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:03.541353 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1288/2563] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:34:03 2022] DELETE /identity/v3/users/97b85a992b8a4a779021d9c2a1542c5b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:34:03.546547 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de70b6c-af4b-4c20-8074-a0179284b098 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:03.547020 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de70b6c-af4b-4c20-8074-a0179284b098 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:03.547137 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de70b6c-af4b-4c20-8074-a0179284b098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:03.565641 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-9de70b6c-af4b-4c20-8074-a0179284b098 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:34:03.566775 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9de70b6c-af4b-4c20-8074-a0179284b098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:03.628514 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1276/2564] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:34:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:34:03.639042 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc65b81-f1f5-4a7c-8f33-34b2f459fd02 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:03.641014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6bc65b81-f1f5-4a7c-8f33-34b2f459fd02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:03.705474 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc65b81-f1f5-4a7c-8f33-34b2f459fd02 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:03.709214 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c49b0b-4322-49d9-80fe-2c4101322e9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:03.709852 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c49b0b-4322-49d9-80fe-2c4101322e9a service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:03.710127 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c49b0b-4322-49d9-80fe-2c4101322e9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:03.710398 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c49b0b-4322-49d9-80fe-2c4101322e9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:03.720247 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c49b0b-4322-49d9-80fe-2c4101322e9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:03.721752 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c49b0b-4322-49d9-80fe-2c4101322e9a service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:03.732428 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1289/2565] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:34:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:34:03.769473 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310ae0b-4797-4e50-ad5f-57110c7f0139 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:03.769762 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310ae0b-4797-4e50-ad5f-57110c7f0139 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:03.770046 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310ae0b-4797-4e50-ad5f-57110c7f0139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:03.794777 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0310ae0b-4797-4e50-ad5f-57110c7f0139 None None] MFA Rules not processed for user `cedaf156c5584d62beedde598c94f322`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:34:03.796527 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0310ae0b-4797-4e50-ad5f-57110c7f0139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:03.861614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1277/2566] 10.222.0.26 () {58 vars in 967 bytes} [Fri Sep 9 20:34:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:34:03.885374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c49b0b-4322-49d9-80fe-2c4101322e9a service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:03.886912 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-01c49b0b-4322-49d9-80fe-2c4101322e9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:03.940249 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:34:03.962173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c49b0b-4322-49d9-80fe-2c4101322e9a service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:03.967176 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08763ab0-d549-4159-b86c-ba159d89eed6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb62d06f7e94e9295c54504ddabb1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:03.968426 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08763ab0-d549-4159-b86c-ba159d89eed6 service swift] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:03.968706 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08763ab0-d549-4159-b86c-ba159d89eed6 service swift] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:03.968977 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08763ab0-d549-4159-b86c-ba159d89eed6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:03.977198 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08763ab0-d549-4159-b86c-ba159d89eed6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:03.978965 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08763ab0-d549-4159-b86c-ba159d89eed6 service swift] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:03.987950 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1290/2567] 10.222.0.26 () {60 vars in 1384 bytes} [Fri Sep 9 20:34:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 20:34:04.015649 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310ae0b-4797-4e50-ad5f-57110c7f0139 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:04.017551 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0310ae0b-4797-4e50-ad5f-57110c7f0139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:04.042233 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310ae0b-4797-4e50-ad5f-57110c7f0139 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:04.051583 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868acbc-8711-4234-b53c-140b81ee9e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:04.052263 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868acbc-8711-4234-b53c-140b81ee9e90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:04.052642 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868acbc-8711-4234-b53c-140b81ee9e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:04.052805 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868acbc-8711-4234-b53c-140b81ee9e90 admin admin] PATH_INFO: `/v3/projects/cc7667a772e6467d8efd6abeb42462c6` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:04.054049 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a868acbc-8711-4234-b53c-140b81ee9e90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc7667a772e6467d8efd6abeb42462c6)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:04.056024 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a868acbc-8711-4234-b53c-140b81ee9e90 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:04.115530 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a868acbc-8711-4234-b53c-140b81ee9e90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc7667a772e6467d8efd6abeb42462c6', 'request_id': 'req-a868acbc-8711-4234-b53c-140b81ee9e90'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:04.121949 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a868acbc-8711-4234-b53c-140b81ee9e90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc7667a772e6467d8efd6abeb42462c6', 'request_id': 'req-a868acbc-8711-4234-b53c-140b81ee9e90'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:04.129806 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a868acbc-8711-4234-b53c-140b81ee9e90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc7667a772e6467d8efd6abeb42462c6', 'request_id': 'req-a868acbc-8711-4234-b53c-140b81ee9e90'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:04.148804 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a868acbc-8711-4234-b53c-140b81ee9e90 admin admin] The token cache is being invalidate because project cc7667a772e6467d8efd6abeb42462c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:34:04.149102 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a868acbc-8711-4234-b53c-140b81ee9e90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:04.149921 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a868acbc-8711-4234-b53c-140b81ee9e90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:04.151509 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1278/2568] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:34:04 2022] DELETE /identity/v3/projects/cc7667a772e6467d8efd6abeb42462c6 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:34:04.156454 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08763ab0-d549-4159-b86c-ba159d89eed6 service swift] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:04.168268 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08763ab0-d549-4159-b86c-ba159d89eed6 service swift] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:04.239295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae1624-177f-4a4d-a3e5-5c9fe2d6fcc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:04.239923 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae1624-177f-4a4d-a3e5-5c9fe2d6fcc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:04.240200 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae1624-177f-4a4d-a3e5-5c9fe2d6fcc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:04.240472 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae1624-177f-4a4d-a3e5-5c9fe2d6fcc1 admin admin] PATH_INFO: `/v3/users/a83a21d5a6e64f2fbc381396b5cebfb7` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:04.241620 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae1624-177f-4a4d-a3e5-5c9fe2d6fcc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a83a21d5a6e64f2fbc381396b5cebfb7)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:04.243777 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae1624-177f-4a4d-a3e5-5c9fe2d6fcc1 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:04.293114 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-e5ae1624-177f-4a4d-a3e5-5c9fe2d6fcc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a83a21d5a6e64f2fbc381396b5cebfb7'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:04.305960 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-e5ae1624-177f-4a4d-a3e5-5c9fe2d6fcc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a83a21d5a6e64f2fbc381396b5cebfb7'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:04.313453 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-e5ae1624-177f-4a4d-a3e5-5c9fe2d6fcc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a83a21d5a6e64f2fbc381396b5cebfb7'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:04.321417 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-e5ae1624-177f-4a4d-a3e5-5c9fe2d6fcc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a83a21d5a6e64f2fbc381396b5cebfb7'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:04.322078 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-e5ae1624-177f-4a4d-a3e5-5c9fe2d6fcc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a83a21d5a6e64f2fbc381396b5cebfb7'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:04.328756 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1291/2569] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:34:04 2022] DELETE /identity/v3/users/a83a21d5a6e64f2fbc381396b5cebfb7 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:34:04.605152 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868acbc-8711-4234-b53c-140b81ee9e90 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:04.606618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a868acbc-8711-4234-b53c-140b81ee9e90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:04.619948 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868acbc-8711-4234-b53c-140b81ee9e90 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:04.628912 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:34:04.679588 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d46fd8-54b1-4b73-86c2-cb9c1766f4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:04.680223 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d46fd8-54b1-4b73-86c2-cb9c1766f4fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:04.680499 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d46fd8-54b1-4b73-86c2-cb9c1766f4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:04.680770 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d46fd8-54b1-4b73-86c2-cb9c1766f4fb admin admin] PATH_INFO: `/v3/projects/b44fee2589724085a5347698fb9ddab8` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:04.681971 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d46fd8-54b1-4b73-86c2-cb9c1766f4fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b44fee2589724085a5347698fb9ddab8)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:04.683888 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d46fd8-54b1-4b73-86c2-cb9c1766f4fb admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:04.729991 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-84d46fd8-54b1-4b73-86c2-cb9c1766f4fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b44fee2589724085a5347698fb9ddab8', 'request_id': 'req-84d46fd8-54b1-4b73-86c2-cb9c1766f4fb'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:04.735496 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-84d46fd8-54b1-4b73-86c2-cb9c1766f4fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b44fee2589724085a5347698fb9ddab8', 'request_id': 'req-84d46fd8-54b1-4b73-86c2-cb9c1766f4fb'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:04.741916 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-84d46fd8-54b1-4b73-86c2-cb9c1766f4fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b44fee2589724085a5347698fb9ddab8', 'request_id': 'req-84d46fd8-54b1-4b73-86c2-cb9c1766f4fb'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:04.754373 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-84d46fd8-54b1-4b73-86c2-cb9c1766f4fb admin admin] The token cache is being invalidate because project b44fee2589724085a5347698fb9ddab8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:34:04.754661 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-84d46fd8-54b1-4b73-86c2-cb9c1766f4fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:04.755301 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-84d46fd8-54b1-4b73-86c2-cb9c1766f4fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:04.756779 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1279/2570] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:34:04 2022] DELETE /identity/v3/projects/b44fee2589724085a5347698fb9ddab8 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:34:04.760986 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae1624-177f-4a4d-a3e5-5c9fe2d6fcc1 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:04.762411 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-e5ae1624-177f-4a4d-a3e5-5c9fe2d6fcc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:04.775487 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae1624-177f-4a4d-a3e5-5c9fe2d6fcc1 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:04.829854 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:34:04.837684 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe3d59-0e00-4321-a3ed-6e2dd7f11d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:04.838543 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe3d59-0e00-4321-a3ed-6e2dd7f11d28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:04.839035 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe3d59-0e00-4321-a3ed-6e2dd7f11d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:04.839479 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe3d59-0e00-4321-a3ed-6e2dd7f11d28 admin admin] PATH_INFO: `/v3/users/d78e923c2181497d8fa13447ce2836d7` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:04.840748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbe3d59-0e00-4321-a3ed-6e2dd7f11d28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d78e923c2181497d8fa13447ce2836d7)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:04.842818 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbe3d59-0e00-4321-a3ed-6e2dd7f11d28 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:04.888807 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-8cbe3d59-0e00-4321-a3ed-6e2dd7f11d28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd78e923c2181497d8fa13447ce2836d7'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:04.900642 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-8cbe3d59-0e00-4321-a3ed-6e2dd7f11d28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd78e923c2181497d8fa13447ce2836d7'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:04.907898 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-8cbe3d59-0e00-4321-a3ed-6e2dd7f11d28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd78e923c2181497d8fa13447ce2836d7'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:04.915362 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-8cbe3d59-0e00-4321-a3ed-6e2dd7f11d28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd78e923c2181497d8fa13447ce2836d7'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:04.916029 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-8cbe3d59-0e00-4321-a3ed-6e2dd7f11d28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd78e923c2181497d8fa13447ce2836d7'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:04.922588 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1292/2571] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:34:04 2022] DELETE /identity/v3/users/d78e923c2181497d8fa13447ce2836d7 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:34:05.190305 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d46fd8-54b1-4b73-86c2-cb9c1766f4fb admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:05.191706 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-84d46fd8-54b1-4b73-86c2-cb9c1766f4fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:05.204032 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d46fd8-54b1-4b73-86c2-cb9c1766f4fb admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:05.267897 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f736e-c68f-4dec-a8d9-20741cd1f906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:05.268543 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f736e-c68f-4dec-a8d9-20741cd1f906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:05.268825 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f736e-c68f-4dec-a8d9-20741cd1f906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:05.269127 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f736e-c68f-4dec-a8d9-20741cd1f906 admin admin] PATH_INFO: `/v3/projects/beeebcf3b7324139b11fef89149b38e0` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:05.270350 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9f736e-c68f-4dec-a8d9-20741cd1f906 admin admin] RBAC: Authorizing `identity:delete_project(project_id=beeebcf3b7324139b11fef89149b38e0)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:05.272266 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9f736e-c68f-4dec-a8d9-20741cd1f906 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:05.315668 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:34:05.319928 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-8f9f736e-c68f-4dec-a8d9-20741cd1f906 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'beeebcf3b7324139b11fef89149b38e0', 'request_id': 'req-8f9f736e-c68f-4dec-a8d9-20741cd1f906'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:05.326331 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-8f9f736e-c68f-4dec-a8d9-20741cd1f906 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'beeebcf3b7324139b11fef89149b38e0', 'request_id': 'req-8f9f736e-c68f-4dec-a8d9-20741cd1f906'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:05.333576 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-8f9f736e-c68f-4dec-a8d9-20741cd1f906 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'beeebcf3b7324139b11fef89149b38e0', 'request_id': 'req-8f9f736e-c68f-4dec-a8d9-20741cd1f906'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:05.348497 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-8f9f736e-c68f-4dec-a8d9-20741cd1f906 admin admin] The token cache is being invalidate because project beeebcf3b7324139b11fef89149b38e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:34:05.348950 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-8f9f736e-c68f-4dec-a8d9-20741cd1f906 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:05.349680 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-8f9f736e-c68f-4dec-a8d9-20741cd1f906 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:05.351231 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1280/2572] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:34:05 2022] DELETE /identity/v3/projects/beeebcf3b7324139b11fef89149b38e0 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:34:05.358126 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3ebdf-1963-46a4-bdc3-d30ad971fc3e None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:05.358491 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3ebdf-1963-46a4-bdc3-d30ad971fc3e None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:05.358853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3ebdf-1963-46a4-bdc3-d30ad971fc3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:05.378643 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-a5a3ebdf-1963-46a4-bdc3-d30ad971fc3e None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:34:05.379832 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a5a3ebdf-1963-46a4-bdc3-d30ad971fc3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:05.438573 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1293/2573] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:34:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:34:05.446808 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55d4e0-c468-47b9-9b51-93330d738c73 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:05.447122 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55d4e0-c468-47b9-9b51-93330d738c73 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:05.447401 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55d4e0-c468-47b9-9b51-93330d738c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:05.466849 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-8c55d4e0-c468-47b9-9b51-93330d738c73 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:34:05.468615 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8c55d4e0-c468-47b9-9b51-93330d738c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:05.476454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1281/2574] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:34:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:34:05.480628 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3ebdf-1963-46a4-bdc3-d30ad971fc3e None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:05.490389 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3ebdf-1963-46a4-bdc3-d30ad971fc3e None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:05.494849 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b0211-2405-436a-a908-c3c328ae7eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:05.495575 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b0211-2405-436a-a908-c3c328ae7eaf admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:05.495867 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b0211-2405-436a-a908-c3c328ae7eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:05.496138 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b0211-2405-436a-a908-c3c328ae7eaf admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:05.496799 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b0211-2405-436a-a908-c3c328ae7eaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:34:05.497098 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b0211-2405-436a-a908-c3c328ae7eaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:05.499249 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b0211-2405-436a-a908-c3c328ae7eaf admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:05.510612 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1294/2575] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:34:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:34:05.515419 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c55d4e0-c468-47b9-9b51-93330d738c73 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:05.524522 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c55d4e0-c468-47b9-9b51-93330d738c73 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:05.529111 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47272d53-1255-4bc3-879e-93188509fdaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:05.529744 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47272d53-1255-4bc3-879e-93188509fdaa admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:05.530054 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47272d53-1255-4bc3-879e-93188509fdaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:05.530332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47272d53-1255-4bc3-879e-93188509fdaa admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:05.530971 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47272d53-1255-4bc3-879e-93188509fdaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:05.532879 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47272d53-1255-4bc3-879e-93188509fdaa admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:05.559051 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1282/2576] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:34:05 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:34:05.563730 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b0211-2405-436a-a908-c3c328ae7eaf admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:05.573412 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b0211-2405-436a-a908-c3c328ae7eaf admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:05.628745 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36068101-dc1a-445c-b7e7-16dfabd5e0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:05.629376 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36068101-dc1a-445c-b7e7-16dfabd5e0f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:05.629657 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36068101-dc1a-445c-b7e7-16dfabd5e0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:05.629930 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36068101-dc1a-445c-b7e7-16dfabd5e0f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:05.630586 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36068101-dc1a-445c-b7e7-16dfabd5e0f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:05.632484 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36068101-dc1a-445c-b7e7-16dfabd5e0f5 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:05.642939 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:34:05.654799 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1295/2577] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:34:05 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:34:05.659146 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47272d53-1255-4bc3-879e-93188509fdaa admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:05.669749 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47272d53-1255-4bc3-879e-93188509fdaa admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:05.674490 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d0cfa1-ffe6-4c02-a13f-ac5d4c3ddb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:05.675123 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d0cfa1-ffe6-4c02-a13f-ac5d4c3ddb28 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:05.675404 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d0cfa1-ffe6-4c02-a13f-ac5d4c3ddb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:05.675681 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d0cfa1-ffe6-4c02-a13f-ac5d4c3ddb28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:05.676372 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d0cfa1-ffe6-4c02-a13f-ac5d4c3ddb28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:05.678035 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d0cfa1-ffe6-4c02-a13f-ac5d4c3ddb28 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:05.688457 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1283/2578] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:34:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:34:05.692494 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36068101-dc1a-445c-b7e7-16dfabd5e0f5 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:05.702834 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36068101-dc1a-445c-b7e7-16dfabd5e0f5 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:05.707370 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ed0ef-99c3-4cee-b269-6c6175b58297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:05.707957 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ed0ef-99c3-4cee-b269-6c6175b58297 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:05.708235 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ed0ef-99c3-4cee-b269-6c6175b58297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:05.708507 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ed0ef-99c3-4cee-b269-6c6175b58297 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:05.709135 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807ed0ef-99c3-4cee-b269-6c6175b58297 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:05.710734 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807ed0ef-99c3-4cee-b269-6c6175b58297 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:05.721032 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1296/2579] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:34:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:34:05.725138 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d0cfa1-ffe6-4c02-a13f-ac5d4c3ddb28 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:05.734790 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d0cfa1-ffe6-4c02-a13f-ac5d4c3ddb28 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:05.739638 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb35ef50-4b7b-46c9-b516-954d96e96371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:05.740274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb35ef50-4b7b-46c9-b516-954d96e96371 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:05.740552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb35ef50-4b7b-46c9-b516-954d96e96371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:05.740822 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb35ef50-4b7b-46c9-b516-954d96e96371 admin admin] PATH_INFO: `/v3/projects/f778994e386643fd97ddc5a700098b63/users/9763f1abc6e0424dbf5e643f4d647496/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:05.755615 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb35ef50-4b7b-46c9-b516-954d96e96371 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f778994e386643fd97ddc5a700098b63, user_id=9763f1abc6e0424dbf5e643f4d647496, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:05.758288 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb35ef50-4b7b-46c9-b516-954d96e96371 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:05.771054 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1284/2580] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:34:05 2022] PUT /identity/v3/projects/f778994e386643fd97ddc5a700098b63/users/9763f1abc6e0424dbf5e643f4d647496/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:34:05.781881 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f2b60-f1b1-4cbf-bf23-ae1f5c79fa21 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:05.782359 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f2b60-f1b1-4cbf-bf23-ae1f5c79fa21 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:05.782456 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f2b60-f1b1-4cbf-bf23-ae1f5c79fa21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:05.808846 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-8d2f2b60-f1b1-4cbf-bf23-ae1f5c79fa21 None None] MFA Rules not processed for user `9763f1abc6e0424dbf5e643f4d647496`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:34:05.810091 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-8d2f2b60-f1b1-4cbf-bf23-ae1f5c79fa21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:05.872406 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1297/2581] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:34:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:34:05.883638 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb35ef50-4b7b-46c9-b516-954d96e96371 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:05.885566 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-cb35ef50-4b7b-46c9-b516-954d96e96371 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:05.942001 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb35ef50-4b7b-46c9-b516-954d96e96371 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:05.945668 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f08315-6672-472f-b920-42ad411be384 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5298486310004b63b51116dbc747e31d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:05.946217 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f08315-6672-472f-b920-42ad411be384 service glance] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:05.946501 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f08315-6672-472f-b920-42ad411be384 service glance] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:05.946777 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f08315-6672-472f-b920-42ad411be384 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:05.955407 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f08315-6672-472f-b920-42ad411be384 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:05.956834 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f08315-6672-472f-b920-42ad411be384 service glance] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:05.965552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1285/2582] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:34:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:34:15.869934 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:INSERT=1,keystone:SELECT=39 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:34:15.961597 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=61,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:34:22.526341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f2b60-f1b1-4cbf-bf23-ae1f5c79fa21 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:22.591158 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f2b60-f1b1-4cbf-bf23-ae1f5c79fa21 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:22.595700 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2412ddb-1971-4d11-9b4f-6bbae57397fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:22.596331 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2412ddb-1971-4d11-9b4f-6bbae57397fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:22.596613 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2412ddb-1971-4d11-9b4f-6bbae57397fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:22.596887 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2412ddb-1971-4d11-9b4f-6bbae57397fd admin admin] PATH_INFO: `/v3/users/9763f1abc6e0424dbf5e643f4d647496` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:22.598038 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2412ddb-1971-4d11-9b4f-6bbae57397fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9763f1abc6e0424dbf5e643f4d647496)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:22.599953 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2412ddb-1971-4d11-9b4f-6bbae57397fd admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:22.646659 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-e2412ddb-1971-4d11-9b4f-6bbae57397fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9763f1abc6e0424dbf5e643f4d647496'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:22.658090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-e2412ddb-1971-4d11-9b4f-6bbae57397fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9763f1abc6e0424dbf5e643f4d647496'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:22.665326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-e2412ddb-1971-4d11-9b4f-6bbae57397fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9763f1abc6e0424dbf5e643f4d647496'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:22.675747 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-e2412ddb-1971-4d11-9b4f-6bbae57397fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9763f1abc6e0424dbf5e643f4d647496'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:22.676372 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-e2412ddb-1971-4d11-9b4f-6bbae57397fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9763f1abc6e0424dbf5e643f4d647496'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:22.682485 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1298/2583] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:34:22 2022] DELETE /identity/v3/users/9763f1abc6e0424dbf5e643f4d647496 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:34:22.687460 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ab651-45b0-4c4b-b3aa-749b8e015750 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:22.687895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ab651-45b0-4c4b-b3aa-749b8e015750 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:22.688062 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ab651-45b0-4c4b-b3aa-749b8e015750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:22.706609 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-4d9ab651-45b0-4c4b-b3aa-749b8e015750 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:34:22.707732 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4d9ab651-45b0-4c4b-b3aa-749b8e015750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:22.770263 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1286/2584] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:34:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:34:22.781590 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2412ddb-1971-4d11-9b4f-6bbae57397fd admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:22.783043 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-e2412ddb-1971-4d11-9b4f-6bbae57397fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:22.838589 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2412ddb-1971-4d11-9b4f-6bbae57397fd admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:22.842711 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ff639-1311-45c5-ba0f-62388861bf23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:22.843313 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ff639-1311-45c5-ba0f-62388861bf23 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:22.843594 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ff639-1311-45c5-ba0f-62388861bf23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:22.843867 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ff639-1311-45c5-ba0f-62388861bf23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:22.852711 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432ff639-1311-45c5-ba0f-62388861bf23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:22.854172 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432ff639-1311-45c5-ba0f-62388861bf23 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:22.864336 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1299/2585] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:34:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:34:23.143206 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9ab651-45b0-4c4b-b3aa-749b8e015750 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:23.145025 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4d9ab651-45b0-4c4b-b3aa-749b8e015750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:23.160741 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9ab651-45b0-4c4b-b3aa-749b8e015750 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:23.166059 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ccb7e-b6ee-47a9-9824-3daf4d0628e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:23.166728 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ccb7e-b6ee-47a9-9824-3daf4d0628e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:23.167005 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ccb7e-b6ee-47a9-9824-3daf4d0628e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:23.167303 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ccb7e-b6ee-47a9-9824-3daf4d0628e7 admin admin] PATH_INFO: `/v3/projects/f778994e386643fd97ddc5a700098b63` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:23.168413 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3ccb7e-b6ee-47a9-9824-3daf4d0628e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f778994e386643fd97ddc5a700098b63)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:23.170381 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3ccb7e-b6ee-47a9-9824-3daf4d0628e7 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:23.215415 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bd3ccb7e-b6ee-47a9-9824-3daf4d0628e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f778994e386643fd97ddc5a700098b63', 'request_id': 'req-bd3ccb7e-b6ee-47a9-9824-3daf4d0628e7'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:23.221422 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bd3ccb7e-b6ee-47a9-9824-3daf4d0628e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f778994e386643fd97ddc5a700098b63', 'request_id': 'req-bd3ccb7e-b6ee-47a9-9824-3daf4d0628e7'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:23.227481 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bd3ccb7e-b6ee-47a9-9824-3daf4d0628e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f778994e386643fd97ddc5a700098b63', 'request_id': 'req-bd3ccb7e-b6ee-47a9-9824-3daf4d0628e7'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:23.239564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bd3ccb7e-b6ee-47a9-9824-3daf4d0628e7 admin admin] The token cache is being invalidate because project f778994e386643fd97ddc5a700098b63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:34:23.239848 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bd3ccb7e-b6ee-47a9-9824-3daf4d0628e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:23.240455 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bd3ccb7e-b6ee-47a9-9824-3daf4d0628e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:23.241916 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1287/2586] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:34:23 2022] DELETE /identity/v3/projects/f778994e386643fd97ddc5a700098b63 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:34:23.246707 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c92c7-56f0-4d62-aa78-cc6d3e022caf None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:23.246993 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c92c7-56f0-4d62-aa78-cc6d3e022caf None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:23.247292 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c92c7-56f0-4d62-aa78-cc6d3e022caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:23.267056 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-6b9c92c7-56f0-4d62-aa78-cc6d3e022caf None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:34:23.268234 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6b9c92c7-56f0-4d62-aa78-cc6d3e022caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:23.284989 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:34:23.328296 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1300/2587] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:34:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:34:23.335935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f652e-7fde-46a0-b5eb-6b044720d46b None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:23.336215 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f652e-7fde-46a0-b5eb-6b044720d46b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:23.336485 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f652e-7fde-46a0-b5eb-6b044720d46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:23.356606 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-5b6f652e-7fde-46a0-b5eb-6b044720d46b None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:34:23.357860 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5b6f652e-7fde-46a0-b5eb-6b044720d46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:23.366220 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1288/2588] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:34:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:34:23.370892 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c92c7-56f0-4d62-aa78-cc6d3e022caf None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:23.381063 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c92c7-56f0-4d62-aa78-cc6d3e022caf None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:23.385778 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcdca7d-1e09-443a-903b-728792ebaadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:23.386461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdca7d-1e09-443a-903b-728792ebaadd admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:23.386737 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdca7d-1e09-443a-903b-728792ebaadd admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:23.387004 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdca7d-1e09-443a-903b-728792ebaadd admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:23.387651 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcdca7d-1e09-443a-903b-728792ebaadd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:34:23.387945 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcdca7d-1e09-443a-903b-728792ebaadd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:23.390182 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcdca7d-1e09-443a-903b-728792ebaadd admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:23.402053 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1301/2589] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:34:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:34:23.407087 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f652e-7fde-46a0-b5eb-6b044720d46b None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:23.416598 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f652e-7fde-46a0-b5eb-6b044720d46b None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:23.421934 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283653c-8002-4996-a04e-6b52131c5710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:23.422534 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283653c-8002-4996-a04e-6b52131c5710 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:23.422849 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283653c-8002-4996-a04e-6b52131c5710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:23.423125 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283653c-8002-4996-a04e-6b52131c5710 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:23.423762 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b283653c-8002-4996-a04e-6b52131c5710 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:23.425689 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b283653c-8002-4996-a04e-6b52131c5710 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:23.452483 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1289/2590] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:34:23 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:34:23.456609 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcdca7d-1e09-443a-903b-728792ebaadd admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:23.466661 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcdca7d-1e09-443a-903b-728792ebaadd admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:23.526823 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80d72fb-17c8-4cc4-81fa-34ff3b3c843d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:23.527507 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d72fb-17c8-4cc4-81fa-34ff3b3c843d admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:23.527687 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d72fb-17c8-4cc4-81fa-34ff3b3c843d admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:23.527956 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d72fb-17c8-4cc4-81fa-34ff3b3c843d admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:23.528586 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80d72fb-17c8-4cc4-81fa-34ff3b3c843d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:23.530570 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80d72fb-17c8-4cc4-81fa-34ff3b3c843d admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:23.550868 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1302/2591] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:34:23 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:34:23.555045 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283653c-8002-4996-a04e-6b52131c5710 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:23.566183 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283653c-8002-4996-a04e-6b52131c5710 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:23.570821 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df1d100-3bbb-4151-bb58-a684fd39b1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:23.571418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1d100-3bbb-4151-bb58-a684fd39b1a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:23.571700 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1d100-3bbb-4151-bb58-a684fd39b1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:23.571973 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1d100-3bbb-4151-bb58-a684fd39b1a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:23.572609 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df1d100-3bbb-4151-bb58-a684fd39b1a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:23.574212 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df1d100-3bbb-4151-bb58-a684fd39b1a5 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:23.584443 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1290/2592] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:34:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:34:23.588336 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80d72fb-17c8-4cc4-81fa-34ff3b3c843d admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:23.598216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80d72fb-17c8-4cc4-81fa-34ff3b3c843d admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:23.602883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f3320f-19f2-4bdf-b7bf-38faf0e6c346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:23.603473 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3320f-19f2-4bdf-b7bf-38faf0e6c346 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:23.603762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3320f-19f2-4bdf-b7bf-38faf0e6c346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:23.604031 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3320f-19f2-4bdf-b7bf-38faf0e6c346 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:23.604666 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f3320f-19f2-4bdf-b7bf-38faf0e6c346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:23.606287 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f3320f-19f2-4bdf-b7bf-38faf0e6c346 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:23.616991 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1303/2593] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:34:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:34:23.620823 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df1d100-3bbb-4151-bb58-a684fd39b1a5 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:23.631104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df1d100-3bbb-4151-bb58-a684fd39b1a5 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:23.635680 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a4636f-23a1-4028-8dd1-090a0c118f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:23.636321 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4636f-23a1-4028-8dd1-090a0c118f65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:23.636598 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4636f-23a1-4028-8dd1-090a0c118f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:23.636873 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4636f-23a1-4028-8dd1-090a0c118f65 admin admin] PATH_INFO: `/v3/projects/01a06ee2ca1a409e811d4facec15588d/users/b3e6a783346b4db486605a0d6f05f107/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:23.651632 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a4636f-23a1-4028-8dd1-090a0c118f65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01a06ee2ca1a409e811d4facec15588d, user_id=b3e6a783346b4db486605a0d6f05f107, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:23.654298 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a4636f-23a1-4028-8dd1-090a0c118f65 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:23.668565 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1291/2594] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:34:23 2022] PUT /identity/v3/projects/01a06ee2ca1a409e811d4facec15588d/users/b3e6a783346b4db486605a0d6f05f107/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:34:23.680690 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a2bba-986f-4403-8c68-47c2223dc7d5 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:23.680980 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a2bba-986f-4403-8c68-47c2223dc7d5 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:23.681267 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a2bba-986f-4403-8c68-47c2223dc7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:23.710880 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-455a2bba-986f-4403-8c68-47c2223dc7d5 None None] MFA Rules not processed for user `b3e6a783346b4db486605a0d6f05f107`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:34:23.712131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-455a2bba-986f-4403-8c68-47c2223dc7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:23.782929 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1304/2595] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:34:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:34:23.790501 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a4636f-23a1-4028-8dd1-090a0c118f65 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:23.792288 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-90a4636f-23a1-4028-8dd1-090a0c118f65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:23.815360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:34:23.854711 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a4636f-23a1-4028-8dd1-090a0c118f65 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:23.858386 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09333e82-f1d8-48e7-932f-41b137bd1fe3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:23.858991 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09333e82-f1d8-48e7-932f-41b137bd1fe3 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:23.859269 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09333e82-f1d8-48e7-932f-41b137bd1fe3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:23.859536 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09333e82-f1d8-48e7-932f-41b137bd1fe3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:23.868838 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09333e82-f1d8-48e7-932f-41b137bd1fe3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:23.870303 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09333e82-f1d8-48e7-932f-41b137bd1fe3 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:23.879457 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1292/2596] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:34:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:34:25.155656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc54ae6-bd6e-49c0-b21f-9c10b1b70955 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:25.156110 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc54ae6-bd6e-49c0-b21f-9c10b1b70955 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:25.156248 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc54ae6-bd6e-49c0-b21f-9c10b1b70955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:25.176132 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-4fc54ae6-bd6e-49c0-b21f-9c10b1b70955 None None] MFA Rules not processed for user `f0eca860c013457ea79e4633f3e6d895`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:34:25.177354 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-4fc54ae6-bd6e-49c0-b21f-9c10b1b70955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:25.189346 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:34:25.234429 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1305/2597] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:34:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:34:25.550058 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09333e82-f1d8-48e7-932f-41b137bd1fe3 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:25.551468 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-09333e82-f1d8-48e7-932f-41b137bd1fe3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:25.610935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09333e82-f1d8-48e7-932f-41b137bd1fe3 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:25.614653 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87927e08-61d5-49a6-8522-83beefae2ba9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:25.615250 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87927e08-61d5-49a6-8522-83beefae2ba9 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:25.615529 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87927e08-61d5-49a6-8522-83beefae2ba9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:25.615794 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87927e08-61d5-49a6-8522-83beefae2ba9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:25.617564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-87927e08-61d5-49a6-8522-83beefae2ba9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:25.627057 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87927e08-61d5-49a6-8522-83beefae2ba9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:25.628458 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87927e08-61d5-49a6-8522-83beefae2ba9 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:25.636571 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1293/2598] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:34:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:34:26.926188 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc54ae6-bd6e-49c0-b21f-9c10b1b70955 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:26.992320 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc54ae6-bd6e-49c0-b21f-9c10b1b70955 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:26.997069 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d01155-deec-4a96-a5bc-324a460e05e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:26.998517 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d01155-deec-4a96-a5bc-324a460e05e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:26.998798 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d01155-deec-4a96-a5bc-324a460e05e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:26.999069 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d01155-deec-4a96-a5bc-324a460e05e8 admin admin] PATH_INFO: `/v3/users/b3e6a783346b4db486605a0d6f05f107` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:27.000228 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d01155-deec-4a96-a5bc-324a460e05e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3e6a783346b4db486605a0d6f05f107)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:27.002165 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d01155-deec-4a96-a5bc-324a460e05e8 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:27.061666 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-d5d01155-deec-4a96-a5bc-324a460e05e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3e6a783346b4db486605a0d6f05f107'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:27.075715 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-d5d01155-deec-4a96-a5bc-324a460e05e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3e6a783346b4db486605a0d6f05f107'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:27.086340 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-d5d01155-deec-4a96-a5bc-324a460e05e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3e6a783346b4db486605a0d6f05f107'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:27.097620 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-d5d01155-deec-4a96-a5bc-324a460e05e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3e6a783346b4db486605a0d6f05f107'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:27.098445 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-d5d01155-deec-4a96-a5bc-324a460e05e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3e6a783346b4db486605a0d6f05f107'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:27.105668 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1306/2599] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:34:26 2022] DELETE /identity/v3/users/b3e6a783346b4db486605a0d6f05f107 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:34:27.111155 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa099c-21d5-459c-a670-36051e183b9d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:27.111648 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa099c-21d5-459c-a670-36051e183b9d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:27.111760 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa099c-21d5-459c-a670-36051e183b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:27.147199 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-d6fa099c-21d5-459c-a670-36051e183b9d None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:34:27.148845 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d6fa099c-21d5-459c-a670-36051e183b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:27.214644 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1294/2600] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:34:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:34:27.248137 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d01155-deec-4a96-a5bc-324a460e05e8 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:27.249663 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-d5d01155-deec-4a96-a5bc-324a460e05e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:27.296117 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:34:27.309005 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d01155-deec-4a96-a5bc-324a460e05e8 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:27.312595 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237c233e-2f17-44eb-aa11-d7442e7b1990 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:27.313209 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c233e-2f17-44eb-aa11-d7442e7b1990 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:27.313532 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c233e-2f17-44eb-aa11-d7442e7b1990 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:27.313920 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c233e-2f17-44eb-aa11-d7442e7b1990 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:27.323257 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237c233e-2f17-44eb-aa11-d7442e7b1990 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:27.324720 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237c233e-2f17-44eb-aa11-d7442e7b1990 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:27.335417 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1307/2601] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:34:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:34:28.124958 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa099c-21d5-459c-a670-36051e183b9d None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:28.126516 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d6fa099c-21d5-459c-a670-36051e183b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:28.143168 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa099c-21d5-459c-a670-36051e183b9d None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:28.156390 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07485157-df7d-4784-a123-6f450f78ad60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:28.162732 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07485157-df7d-4784-a123-6f450f78ad60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:28.163307 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07485157-df7d-4784-a123-6f450f78ad60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:28.166521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07485157-df7d-4784-a123-6f450f78ad60 admin admin] PATH_INFO: `/v3/projects/01a06ee2ca1a409e811d4facec15588d` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:28.168520 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07485157-df7d-4784-a123-6f450f78ad60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01a06ee2ca1a409e811d4facec15588d)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:28.171028 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07485157-df7d-4784-a123-6f450f78ad60 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:28.249509 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-07485157-df7d-4784-a123-6f450f78ad60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01a06ee2ca1a409e811d4facec15588d', 'request_id': 'req-07485157-df7d-4784-a123-6f450f78ad60'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:28.256010 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-07485157-df7d-4784-a123-6f450f78ad60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01a06ee2ca1a409e811d4facec15588d', 'request_id': 'req-07485157-df7d-4784-a123-6f450f78ad60'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:28.258711 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:34:28.264812 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-07485157-df7d-4784-a123-6f450f78ad60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01a06ee2ca1a409e811d4facec15588d', 'request_id': 'req-07485157-df7d-4784-a123-6f450f78ad60'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:28.283580 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-07485157-df7d-4784-a123-6f450f78ad60 admin admin] The token cache is being invalidate because project 01a06ee2ca1a409e811d4facec15588d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:34:28.284062 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-07485157-df7d-4784-a123-6f450f78ad60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:28.284722 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-07485157-df7d-4784-a123-6f450f78ad60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:28.286497 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1295/2602] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:34:28 2022] DELETE /identity/v3/projects/01a06ee2ca1a409e811d4facec15588d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:34:28.291572 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314626eb-5044-4843-a285-cc2deea56d0b None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:28.291988 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314626eb-5044-4843-a285-cc2deea56d0b None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:28.292124 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314626eb-5044-4843-a285-cc2deea56d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:28.323750 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-314626eb-5044-4843-a285-cc2deea56d0b None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:34:28.325415 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-314626eb-5044-4843-a285-cc2deea56d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:28.400400 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1308/2603] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:34:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:34:28.410520 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df6b0b-f418-48ba-ab2d-221ba028137f None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:28.410520 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df6b0b-f418-48ba-ab2d-221ba028137f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:28.410520 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df6b0b-f418-48ba-ab2d-221ba028137f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:28.432954 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-c7df6b0b-f418-48ba-ab2d-221ba028137f None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:34:28.434280 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c7df6b0b-f418-48ba-ab2d-221ba028137f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:28.448360 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1296/2604] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:34:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:34:28.449717 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314626eb-5044-4843-a285-cc2deea56d0b None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:28.459869 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314626eb-5044-4843-a285-cc2deea56d0b None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:28.464488 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b48c1ac-4276-4118-9c03-c964cd6816a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:28.465176 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48c1ac-4276-4118-9c03-c964cd6816a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:28.465597 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48c1ac-4276-4118-9c03-c964cd6816a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:28.465937 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48c1ac-4276-4118-9c03-c964cd6816a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:28.466687 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b48c1ac-4276-4118-9c03-c964cd6816a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:34:28.467051 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b48c1ac-4276-4118-9c03-c964cd6816a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:28.468759 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b48c1ac-4276-4118-9c03-c964cd6816a8 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:28.484349 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1309/2605] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:34:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:34:28.485971 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df6b0b-f418-48ba-ab2d-221ba028137f None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:28.495226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df6b0b-f418-48ba-ab2d-221ba028137f None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:28.500619 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ca4f7-982f-469e-b9d9-4710b165d0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:28.501229 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ca4f7-982f-469e-b9d9-4710b165d0aa admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:28.501538 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ca4f7-982f-469e-b9d9-4710b165d0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:28.501812 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ca4f7-982f-469e-b9d9-4710b165d0aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:28.502451 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ca4f7-982f-469e-b9d9-4710b165d0aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:28.504336 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ca4f7-982f-469e-b9d9-4710b165d0aa admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:28.533708 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1297/2606] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:34:28 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:34:28.537989 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b48c1ac-4276-4118-9c03-c964cd6816a8 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:28.548483 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b48c1ac-4276-4118-9c03-c964cd6816a8 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:28.608430 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb1e44-ab0a-4606-adf6-4784b9e2873b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:28.609320 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb1e44-ab0a-4606-adf6-4784b9e2873b admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:28.609601 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb1e44-ab0a-4606-adf6-4784b9e2873b admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:28.609921 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb1e44-ab0a-4606-adf6-4784b9e2873b admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:28.610503 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb1e44-ab0a-4606-adf6-4784b9e2873b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:28.612703 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb1e44-ab0a-4606-adf6-4784b9e2873b admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:28.634814 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1310/2607] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:34:28 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:34:28.639009 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ca4f7-982f-469e-b9d9-4710b165d0aa admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:28.649348 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ca4f7-982f-469e-b9d9-4710b165d0aa admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:28.654679 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b5291-c0b8-451f-ae0a-e552ded22d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:28.655292 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b5291-c0b8-451f-ae0a-e552ded22d6c admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:28.660272 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b5291-c0b8-451f-ae0a-e552ded22d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:28.660272 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b5291-c0b8-451f-ae0a-e552ded22d6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:28.660272 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3b5291-c0b8-451f-ae0a-e552ded22d6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:28.660272 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3b5291-c0b8-451f-ae0a-e552ded22d6c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:28.669062 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1298/2608] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:34:28.675692 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb1e44-ab0a-4606-adf6-4784b9e2873b admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:28.686566 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb1e44-ab0a-4606-adf6-4784b9e2873b admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:28.691496 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9054f6b7-3ae1-415f-970b-2dc8567b780a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:28.692164 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054f6b7-3ae1-415f-970b-2dc8567b780a admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:28.692557 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054f6b7-3ae1-415f-970b-2dc8567b780a admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:28.692829 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054f6b7-3ae1-415f-970b-2dc8567b780a admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:28.693490 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9054f6b7-3ae1-415f-970b-2dc8567b780a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:28.695088 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9054f6b7-3ae1-415f-970b-2dc8567b780a admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:28.706263 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1311/2609] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:34:28.710364 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b5291-c0b8-451f-ae0a-e552ded22d6c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:28.720956 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b5291-c0b8-451f-ae0a-e552ded22d6c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:28.725744 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a15017-9f85-4bc5-8e03-1d433a186c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:28.726401 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a15017-9f85-4bc5-8e03-1d433a186c5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:28.726686 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a15017-9f85-4bc5-8e03-1d433a186c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:28.726963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a15017-9f85-4bc5-8e03-1d433a186c5e admin admin] PATH_INFO: `/v3/projects/9d37e9b0f6e6404882c0c7d4b25fd04b/users/7176d1895248400887578343dfef6a77/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:28.743152 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a15017-9f85-4bc5-8e03-1d433a186c5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d37e9b0f6e6404882c0c7d4b25fd04b, user_id=7176d1895248400887578343dfef6a77, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:28.746466 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a15017-9f85-4bc5-8e03-1d433a186c5e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:28.764774 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1299/2610] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:34:28 2022] PUT /identity/v3/projects/9d37e9b0f6e6404882c0c7d4b25fd04b/users/7176d1895248400887578343dfef6a77/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:34:28.777607 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ed9be-fbad-4567-b556-4efc7113633b None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:28.777903 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ed9be-fbad-4567-b556-4efc7113633b None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:28.778259 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ed9be-fbad-4567-b556-4efc7113633b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:28.812938 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-b89ed9be-fbad-4567-b556-4efc7113633b None None] MFA Rules not processed for user `7176d1895248400887578343dfef6a77`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:34:28.814211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-b89ed9be-fbad-4567-b556-4efc7113633b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:28.856729 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:34:28.883977 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1312/2611] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:34:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:34:28.895339 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a15017-9f85-4bc5-8e03-1d433a186c5e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:28.897070 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f5a15017-9f85-4bc5-8e03-1d433a186c5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:28.976094 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a15017-9f85-4bc5-8e03-1d433a186c5e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:28.980140 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8329be72-c1fb-4275-a149-db02782aafcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:28.980754 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329be72-c1fb-4275-a149-db02782aafcb service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:28.981046 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329be72-c1fb-4275-a149-db02782aafcb service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:28.981331 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329be72-c1fb-4275-a149-db02782aafcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:28.990391 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8329be72-c1fb-4275-a149-db02782aafcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:28.991804 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8329be72-c1fb-4275-a149-db02782aafcb service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:29.001353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1300/2612] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:34:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:34:36.474986 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ed9be-fbad-4567-b556-4efc7113633b None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:36.476401 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-b89ed9be-fbad-4567-b556-4efc7113633b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:36.549326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ed9be-fbad-4567-b556-4efc7113633b None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:36.554731 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f06bd-9359-4dd2-92d6-5b8b4760b06f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:36.555302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f06bd-9359-4dd2-92d6-5b8b4760b06f service nova] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:36.555576 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f06bd-9359-4dd2-92d6-5b8b4760b06f service nova] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:36.555845 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f06bd-9359-4dd2-92d6-5b8b4760b06f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:36.557562 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-8d0f06bd-9359-4dd2-92d6-5b8b4760b06f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:36.570170 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f06bd-9359-4dd2-92d6-5b8b4760b06f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:36.571743 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f06bd-9359-4dd2-92d6-5b8b4760b06f service nova] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:36.582902 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1313/2613] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:34:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:34:38.996570 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=56,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:34:46.578263 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=43 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:34:50.659924 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8329be72-c1fb-4275-a149-db02782aafcb service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:50.661423 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8329be72-c1fb-4275-a149-db02782aafcb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:50.721834 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8329be72-c1fb-4275-a149-db02782aafcb service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:50.725448 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb387b6-a51f-45b9-a827-da9cf58b0c17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:50.726057 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb387b6-a51f-45b9-a827-da9cf58b0c17 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:50.726337 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb387b6-a51f-45b9-a827-da9cf58b0c17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:50.726610 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb387b6-a51f-45b9-a827-da9cf58b0c17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:50.728348 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5eb387b6-a51f-45b9-a827-da9cf58b0c17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:50.792552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb387b6-a51f-45b9-a827-da9cf58b0c17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:50.794617 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb387b6-a51f-45b9-a827-da9cf58b0c17 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:50.804553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1301/2614] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:34:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:34:54.014202 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f06bd-9359-4dd2-92d6-5b8b4760b06f service nova] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:54.015736 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-8d0f06bd-9359-4dd2-92d6-5b8b4760b06f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:54.032951 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f06bd-9359-4dd2-92d6-5b8b4760b06f service nova] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:54.037377 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e74a5-3276-4bbc-a02a-1130e0da676b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:54.038038 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e74a5-3276-4bbc-a02a-1130e0da676b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:54.038319 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e74a5-3276-4bbc-a02a-1130e0da676b admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:54.038594 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e74a5-3276-4bbc-a02a-1130e0da676b admin admin] PATH_INFO: `/v3/users/db1f365ebf794941b3999adb37c48100` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:54.039719 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e74a5-3276-4bbc-a02a-1130e0da676b admin admin] RBAC: Authorizing `identity:delete_user(user_id=db1f365ebf794941b3999adb37c48100)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:54.041672 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e74a5-3276-4bbc-a02a-1130e0da676b admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:54.094450 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-f98e74a5-3276-4bbc-a02a-1130e0da676b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db1f365ebf794941b3999adb37c48100'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:54.116182 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-f98e74a5-3276-4bbc-a02a-1130e0da676b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db1f365ebf794941b3999adb37c48100'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:54.130212 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-f98e74a5-3276-4bbc-a02a-1130e0da676b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db1f365ebf794941b3999adb37c48100'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:54.138082 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-f98e74a5-3276-4bbc-a02a-1130e0da676b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db1f365ebf794941b3999adb37c48100'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:54.138884 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-f98e74a5-3276-4bbc-a02a-1130e0da676b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db1f365ebf794941b3999adb37c48100'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:54.150036 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1314/2615] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:34:54 2022] DELETE /identity/v3/users/db1f365ebf794941b3999adb37c48100 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:34:54.324694 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb387b6-a51f-45b9-a827-da9cf58b0c17 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:54.335055 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb387b6-a51f-45b9-a827-da9cf58b0c17 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:54.392980 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0289a4e-ac15-440d-aca0-2b3480ef30da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:54.393666 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0289a4e-ac15-440d-aca0-2b3480ef30da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:54.393944 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0289a4e-ac15-440d-aca0-2b3480ef30da admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:54.394212 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0289a4e-ac15-440d-aca0-2b3480ef30da admin admin] PATH_INFO: `/v3/projects/8c1c909559e44322a07f7e4a463424a5` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:54.395323 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0289a4e-ac15-440d-aca0-2b3480ef30da admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c1c909559e44322a07f7e4a463424a5)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:54.397202 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0289a4e-ac15-440d-aca0-2b3480ef30da admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:54.446205 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b0289a4e-ac15-440d-aca0-2b3480ef30da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c1c909559e44322a07f7e4a463424a5', 'request_id': 'req-b0289a4e-ac15-440d-aca0-2b3480ef30da'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:54.448085 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:34:54.452568 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b0289a4e-ac15-440d-aca0-2b3480ef30da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c1c909559e44322a07f7e4a463424a5', 'request_id': 'req-b0289a4e-ac15-440d-aca0-2b3480ef30da'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:54.459264 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b0289a4e-ac15-440d-aca0-2b3480ef30da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c1c909559e44322a07f7e4a463424a5', 'request_id': 'req-b0289a4e-ac15-440d-aca0-2b3480ef30da'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:54.471875 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b0289a4e-ac15-440d-aca0-2b3480ef30da admin admin] The token cache is being invalidate because project 8c1c909559e44322a07f7e4a463424a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:34:54.472164 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b0289a4e-ac15-440d-aca0-2b3480ef30da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:54.472788 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b0289a4e-ac15-440d-aca0-2b3480ef30da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:54.474337 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1302/2616] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:34:54 2022] DELETE /identity/v3/projects/8c1c909559e44322a07f7e4a463424a5 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:34:54.479318 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1b223-f35e-4091-ad45-506593bbb785 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:54.479627 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1b223-f35e-4091-ad45-506593bbb785 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:54.479897 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1b223-f35e-4091-ad45-506593bbb785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:54.501073 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-5cf1b223-f35e-4091-ad45-506593bbb785 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:34:54.502299 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-5cf1b223-f35e-4091-ad45-506593bbb785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:54.563854 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1315/2617] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:34:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:34:54.571590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64051a47-446f-43e2-b6f9-32a34475361e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:54.571875 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64051a47-446f-43e2-b6f9-32a34475361e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:54.572144 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64051a47-446f-43e2-b6f9-32a34475361e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:54.592895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-64051a47-446f-43e2-b6f9-32a34475361e None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:34:54.594132 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-64051a47-446f-43e2-b6f9-32a34475361e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:54.602152 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1303/2618] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:34:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:34:54.606255 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf1b223-f35e-4091-ad45-506593bbb785 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:54.616465 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf1b223-f35e-4091-ad45-506593bbb785 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:54.621298 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ee243-e9ed-41c7-9ff8-cb439f7f3e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:54.622094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ee243-e9ed-41c7-9ff8-cb439f7f3e15 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:54.622637 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ee243-e9ed-41c7-9ff8-cb439f7f3e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:54.622933 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ee243-e9ed-41c7-9ff8-cb439f7f3e15 admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:54.623649 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ee243-e9ed-41c7-9ff8-cb439f7f3e15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:34:54.623940 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ee243-e9ed-41c7-9ff8-cb439f7f3e15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:54.625807 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ee243-e9ed-41c7-9ff8-cb439f7f3e15 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:54.638195 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1316/2619] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:34:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:34:54.642557 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64051a47-446f-43e2-b6f9-32a34475361e None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:54.652016 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64051a47-446f-43e2-b6f9-32a34475361e None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:54.662473 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca7f0a3-9463-449b-b44a-31c4b8447e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:54.663085 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca7f0a3-9463-449b-b44a-31c4b8447e6b admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:54.663362 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca7f0a3-9463-449b-b44a-31c4b8447e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:54.663637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca7f0a3-9463-449b-b44a-31c4b8447e6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:54.664333 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca7f0a3-9463-449b-b44a-31c4b8447e6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:54.666262 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca7f0a3-9463-449b-b44a-31c4b8447e6b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:54.694669 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1304/2620] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:34:54 2022] POST /identity/v3/projects => generated 403 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:34:54.698750 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ee243-e9ed-41c7-9ff8-cb439f7f3e15 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:54.709095 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ee243-e9ed-41c7-9ff8-cb439f7f3e15 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:54.767148 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be8b446-90b0-4acf-83ed-f6f2d262c420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:54.767740 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8b446-90b0-4acf-83ed-f6f2d262c420 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:54.768018 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8b446-90b0-4acf-83ed-f6f2d262c420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:54.768304 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8b446-90b0-4acf-83ed-f6f2d262c420 admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:54.768902 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be8b446-90b0-4acf-83ed-f6f2d262c420 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:54.770876 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be8b446-90b0-4acf-83ed-f6f2d262c420 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:54.779445 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:34:54.793771 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1317/2621] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:34:54 2022] POST /identity/v3/users => generated 302 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:34:54.797869 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca7f0a3-9463-449b-b44a-31c4b8447e6b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:54.814647 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca7f0a3-9463-449b-b44a-31c4b8447e6b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:54.819393 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd9e0b-eb42-4af4-8968-f76d8812b43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:54.820009 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd9e0b-eb42-4af4-8968-f76d8812b43e admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:54.820306 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd9e0b-eb42-4af4-8968-f76d8812b43e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:54.820586 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd9e0b-eb42-4af4-8968-f76d8812b43e admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:54.821210 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd9e0b-eb42-4af4-8968-f76d8812b43e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:54.823222 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd9e0b-eb42-4af4-8968-f76d8812b43e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:54.833839 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1305/2622] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:34:54.837790 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be8b446-90b0-4acf-83ed-f6f2d262c420 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:54.848324 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be8b446-90b0-4acf-83ed-f6f2d262c420 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:54.852923 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde71d16-0192-4edf-a4d9-9fe1b7b2c475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:54.853559 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde71d16-0192-4edf-a4d9-9fe1b7b2c475 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:54.853848 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde71d16-0192-4edf-a4d9-9fe1b7b2c475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:54.854126 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde71d16-0192-4edf-a4d9-9fe1b7b2c475 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:54.854764 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde71d16-0192-4edf-a4d9-9fe1b7b2c475 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:54.856387 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde71d16-0192-4edf-a4d9-9fe1b7b2c475 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:54.867204 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1318/2623] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:34:54.871396 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd9e0b-eb42-4af4-8968-f76d8812b43e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:54.881408 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd9e0b-eb42-4af4-8968-f76d8812b43e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:54.886018 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f4d341-02f4-4a06-bfa3-01ccac41aa1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:54.886662 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4d341-02f4-4a06-bfa3-01ccac41aa1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:54.886943 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4d341-02f4-4a06-bfa3-01ccac41aa1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:54.887718 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4d341-02f4-4a06-bfa3-01ccac41aa1e admin admin] PATH_INFO: `/v3/projects/122f88e06223456eb1edc6b93c00a1d7/users/125f5c5468f74fffafdb08f0d68f2903/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:54.902092 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f4d341-02f4-4a06-bfa3-01ccac41aa1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=122f88e06223456eb1edc6b93c00a1d7, user_id=125f5c5468f74fffafdb08f0d68f2903, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:54.905354 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f4d341-02f4-4a06-bfa3-01ccac41aa1e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:54.917554 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1306/2624] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:34:54 2022] PUT /identity/v3/projects/122f88e06223456eb1edc6b93c00a1d7/users/125f5c5468f74fffafdb08f0d68f2903/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:34:54.922554 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad096d64-727e-4330-82cd-dedde7b33527 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:54.922844 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad096d64-727e-4330-82cd-dedde7b33527 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:54.923123 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad096d64-727e-4330-82cd-dedde7b33527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:54.941758 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-ad096d64-727e-4330-82cd-dedde7b33527 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:34:54.942947 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-ad096d64-727e-4330-82cd-dedde7b33527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:55.000867 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1319/2625] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:34:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:34:55.010679 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f4d341-02f4-4a06-bfa3-01ccac41aa1e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:55.012417 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-01f4d341-02f4-4a06-bfa3-01ccac41aa1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:55.075764 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f4d341-02f4-4a06-bfa3-01ccac41aa1e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:55.078328 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdacd85-b3f4-452a-bb3c-7c523d3eb6aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:55.079085 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdacd85-b3f4-452a-bb3c-7c523d3eb6aa service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:55.079432 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdacd85-b3f4-452a-bb3c-7c523d3eb6aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:55.079858 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdacd85-b3f4-452a-bb3c-7c523d3eb6aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:55.091709 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdacd85-b3f4-452a-bb3c-7c523d3eb6aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:55.093207 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdacd85-b3f4-452a-bb3c-7c523d3eb6aa service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:55.104826 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1307/2626] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:34:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:34:55.723948 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3fee6-4ed6-40ec-b4d8-09f4e90a6e04 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:55.724453 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3fee6-4ed6-40ec-b4d8-09f4e90a6e04 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:55.724575 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3fee6-4ed6-40ec-b4d8-09f4e90a6e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:55.754021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-70e3fee6-4ed6-40ec-b4d8-09f4e90a6e04 None None] MFA Rules not processed for user `125f5c5468f74fffafdb08f0d68f2903`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:34:55.755237 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-70e3fee6-4ed6-40ec-b4d8-09f4e90a6e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:55.829639 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1320/2627] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:34:55 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:34:55.834222 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdacd85-b3f4-452a-bb3c-7c523d3eb6aa service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:55.844269 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdacd85-b3f4-452a-bb3c-7c523d3eb6aa service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:55.850194 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15912d4e-b121-49de-a034-d677c0d84c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:55.850815 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15912d4e-b121-49de-a034-d677c0d84c16 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:55.851117 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15912d4e-b121-49de-a034-d677c0d84c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:55.851394 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15912d4e-b121-49de-a034-d677c0d84c16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:55.852095 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15912d4e-b121-49de-a034-d677c0d84c16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:55.853946 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15912d4e-b121-49de-a034-d677c0d84c16 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:55.885203 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1308/2628] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:34:55 2022] POST /identity/v3/projects => generated 405 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:34:55.886360 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e3fee6-4ed6-40ec-b4d8-09f4e90a6e04 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:55.899240 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e3fee6-4ed6-40ec-b4d8-09f4e90a6e04 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:55.956214 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7348bce-e108-4839-8479-097cc04b07c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:55.956741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7348bce-e108-4839-8479-097cc04b07c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:55.957019 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7348bce-e108-4839-8479-097cc04b07c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:55.957316 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7348bce-e108-4839-8479-097cc04b07c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:55.957928 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7348bce-e108-4839-8479-097cc04b07c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:55.959823 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7348bce-e108-4839-8479-097cc04b07c6 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:55.975725 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:34:55.982922 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1321/2629] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:34:55 2022] POST /identity/v3/users => generated 309 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:34:55.987272 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15912d4e-b121-49de-a034-d677c0d84c16 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:55.997900 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15912d4e-b121-49de-a034-d677c0d84c16 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:56.002585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0078686-d5e4-4782-9337-2906b76b2ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:56.003252 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0078686-d5e4-4782-9337-2906b76b2ae8 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:56.003634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0078686-d5e4-4782-9337-2906b76b2ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:56.003881 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0078686-d5e4-4782-9337-2906b76b2ae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:56.004541 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0078686-d5e4-4782-9337-2906b76b2ae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:56.006136 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0078686-d5e4-4782-9337-2906b76b2ae8 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:56.016897 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1309/2630] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:34:56.021027 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7348bce-e108-4839-8479-097cc04b07c6 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:56.031463 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7348bce-e108-4839-8479-097cc04b07c6 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:56.036224 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee1b591-c8bf-4b6f-a6f2-3903306ff064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:56.036868 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1b591-c8bf-4b6f-a6f2-3903306ff064 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:56.037145 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1b591-c8bf-4b6f-a6f2-3903306ff064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:56.037442 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1b591-c8bf-4b6f-a6f2-3903306ff064 admin admin] PATH_INFO: `/v3/projects/8607f205aafa4e849e534753fdf74cf4/users/d5782b18313545338ca038abb0f83a3e/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:56.051982 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee1b591-c8bf-4b6f-a6f2-3903306ff064 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8607f205aafa4e849e534753fdf74cf4, user_id=d5782b18313545338ca038abb0f83a3e, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:56.054618 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee1b591-c8bf-4b6f-a6f2-3903306ff064 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:56.067798 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1322/2631] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:34:56 2022] PUT /identity/v3/projects/8607f205aafa4e849e534753fdf74cf4/users/d5782b18313545338ca038abb0f83a3e/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:34:56.071820 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0078686-d5e4-4782-9337-2906b76b2ae8 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:56.081880 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0078686-d5e4-4782-9337-2906b76b2ae8 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:56.125499 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:34:56.140509 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344a532-118a-426c-a557-c53c646438fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:56.140970 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344a532-118a-426c-a557-c53c646438fb admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:56.141267 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344a532-118a-426c-a557-c53c646438fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:56.141569 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344a532-118a-426c-a557-c53c646438fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:56.142509 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1344a532-118a-426c-a557-c53c646438fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:56.143841 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1344a532-118a-426c-a557-c53c646438fb admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:56.157880 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1310/2632] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:34:56.158817 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee1b591-c8bf-4b6f-a6f2-3903306ff064 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:56.169648 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee1b591-c8bf-4b6f-a6f2-3903306ff064 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:56.174432 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba51ef-4ed8-47b8-bb74-f3b694c1bd1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:56.175140 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba51ef-4ed8-47b8-bb74-f3b694c1bd1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:56.175467 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba51ef-4ed8-47b8-bb74-f3b694c1bd1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:56.175744 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba51ef-4ed8-47b8-bb74-f3b694c1bd1a admin admin] PATH_INFO: `/v3/projects/8607f205aafa4e849e534753fdf74cf4/users/d5782b18313545338ca038abb0f83a3e/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:56.177672 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aba51ef-4ed8-47b8-bb74-f3b694c1bd1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8607f205aafa4e849e534753fdf74cf4, user_id=d5782b18313545338ca038abb0f83a3e, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:56.180277 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aba51ef-4ed8-47b8-bb74-f3b694c1bd1a admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:56.190753 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1323/2633] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:34:56 2022] PUT /identity/v3/projects/8607f205aafa4e849e534753fdf74cf4/users/d5782b18313545338ca038abb0f83a3e/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:34:57.043044 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86222d-c8c3-4b6d-9562-c388b89420d0 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:57.043592 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86222d-c8c3-4b6d-9562-c388b89420d0 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:57.043592 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86222d-c8c3-4b6d-9562-c388b89420d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:57.072365 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0f86222d-c8c3-4b6d-9562-c388b89420d0 None None] MFA Rules not processed for user `d5782b18313545338ca038abb0f83a3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:34:57.073620 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0f86222d-c8c3-4b6d-9562-c388b89420d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:57.146240 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1311/2634] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:34:57 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:34:58.092451 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba51ef-4ed8-47b8-bb74-f3b694c1bd1a admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:58.094197 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7aba51ef-4ed8-47b8-bb74-f3b694c1bd1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:58.165367 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba51ef-4ed8-47b8-bb74-f3b694c1bd1a admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:58.170057 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86222d-c8c3-4b6d-9562-c388b89420d0 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:58.172028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc7b13-0c3d-4bb9-8f61-d9f1c9b2e597 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26d24cf0a21d491082b0c2f11e749446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:58.173169 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc7b13-0c3d-4bb9-8f61-d9f1c9b2e597 service placement] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:58.173487 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc7b13-0c3d-4bb9-8f61-d9f1c9b2e597 service placement] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:58.173765 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc7b13-0c3d-4bb9-8f61-d9f1c9b2e597 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:58.175650 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-52fc7b13-0c3d-4bb9-8f61-d9f1c9b2e597 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:58.187516 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fc7b13-0c3d-4bb9-8f61-d9f1c9b2e597 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:58.189124 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fc7b13-0c3d-4bb9-8f61-d9f1c9b2e597 service placement] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:58.199034 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1324/2635] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:34:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:34:58.236897 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86222d-c8c3-4b6d-9562-c388b89420d0 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:58.241539 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2541f4e-e0f0-4ee2-9d8d-ccdcc64c0ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:58.242172 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2541f4e-e0f0-4ee2-9d8d-ccdcc64c0ae4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:58.242455 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2541f4e-e0f0-4ee2-9d8d-ccdcc64c0ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:58.242730 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2541f4e-e0f0-4ee2-9d8d-ccdcc64c0ae4 admin admin] PATH_INFO: `/v3/users/125f5c5468f74fffafdb08f0d68f2903` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:58.243851 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2541f4e-e0f0-4ee2-9d8d-ccdcc64c0ae4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=125f5c5468f74fffafdb08f0d68f2903)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:58.245792 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2541f4e-e0f0-4ee2-9d8d-ccdcc64c0ae4 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:58.297562 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b2541f4e-e0f0-4ee2-9d8d-ccdcc64c0ae4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '125f5c5468f74fffafdb08f0d68f2903'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:58.304516 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b2541f4e-e0f0-4ee2-9d8d-ccdcc64c0ae4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '125f5c5468f74fffafdb08f0d68f2903'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:58.315082 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:34:58.317659 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b2541f4e-e0f0-4ee2-9d8d-ccdcc64c0ae4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '125f5c5468f74fffafdb08f0d68f2903'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:58.325436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b2541f4e-e0f0-4ee2-9d8d-ccdcc64c0ae4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '125f5c5468f74fffafdb08f0d68f2903'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:58.326034 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b2541f4e-e0f0-4ee2-9d8d-ccdcc64c0ae4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '125f5c5468f74fffafdb08f0d68f2903'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:58.331848 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1312/2636] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:34:58 2022] DELETE /identity/v3/users/125f5c5468f74fffafdb08f0d68f2903 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:34:58.502452 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc7b13-0c3d-4bb9-8f61-d9f1c9b2e597 service placement] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:58.516003 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc7b13-0c3d-4bb9-8f61-d9f1c9b2e597 service placement] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:58.572136 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189de99d-de13-4ff7-95e3-33931148365b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:58.572873 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189de99d-de13-4ff7-95e3-33931148365b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:58.573157 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189de99d-de13-4ff7-95e3-33931148365b admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:58.573455 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189de99d-de13-4ff7-95e3-33931148365b admin admin] PATH_INFO: `/v3/projects/122f88e06223456eb1edc6b93c00a1d7` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:58.574582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189de99d-de13-4ff7-95e3-33931148365b admin admin] RBAC: Authorizing `identity:delete_project(project_id=122f88e06223456eb1edc6b93c00a1d7)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:58.576460 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189de99d-de13-4ff7-95e3-33931148365b admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:58.623681 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-189de99d-de13-4ff7-95e3-33931148365b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '122f88e06223456eb1edc6b93c00a1d7', 'request_id': 'req-189de99d-de13-4ff7-95e3-33931148365b'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:58.626354 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:34:58.631486 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-189de99d-de13-4ff7-95e3-33931148365b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '122f88e06223456eb1edc6b93c00a1d7', 'request_id': 'req-189de99d-de13-4ff7-95e3-33931148365b'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:58.638898 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-189de99d-de13-4ff7-95e3-33931148365b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '122f88e06223456eb1edc6b93c00a1d7', 'request_id': 'req-189de99d-de13-4ff7-95e3-33931148365b'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:58.650625 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-189de99d-de13-4ff7-95e3-33931148365b admin admin] The token cache is being invalidate because project 122f88e06223456eb1edc6b93c00a1d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:34:58.650958 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-189de99d-de13-4ff7-95e3-33931148365b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:58.651603 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-189de99d-de13-4ff7-95e3-33931148365b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:58.653075 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1325/2637] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:34:58 2022] DELETE /identity/v3/projects/122f88e06223456eb1edc6b93c00a1d7 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:34:58.657274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2541f4e-e0f0-4ee2-9d8d-ccdcc64c0ae4 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:58.658911 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b2541f4e-e0f0-4ee2-9d8d-ccdcc64c0ae4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:58.673973 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2541f4e-e0f0-4ee2-9d8d-ccdcc64c0ae4 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:58.736590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a74ef-b7af-4d23-b743-198f1b462dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:58.737537 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a74ef-b7af-4d23-b743-198f1b462dcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:58.737822 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a74ef-b7af-4d23-b743-198f1b462dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:58.738129 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a74ef-b7af-4d23-b743-198f1b462dcd admin admin] PATH_INFO: `/v3/users/d5782b18313545338ca038abb0f83a3e` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:58.739519 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a74ef-b7af-4d23-b743-198f1b462dcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5782b18313545338ca038abb0f83a3e)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:58.741481 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a74ef-b7af-4d23-b743-198f1b462dcd admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:58.792160 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b02a74ef-b7af-4d23-b743-198f1b462dcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5782b18313545338ca038abb0f83a3e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:58.802537 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b02a74ef-b7af-4d23-b743-198f1b462dcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5782b18313545338ca038abb0f83a3e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:58.814913 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b02a74ef-b7af-4d23-b743-198f1b462dcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5782b18313545338ca038abb0f83a3e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:58.821556 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b02a74ef-b7af-4d23-b743-198f1b462dcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5782b18313545338ca038abb0f83a3e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:58.822174 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b02a74ef-b7af-4d23-b743-198f1b462dcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5782b18313545338ca038abb0f83a3e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:58.828226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1313/2638] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:34:58 2022] DELETE /identity/v3/users/d5782b18313545338ca038abb0f83a3e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:34:59.001111 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189de99d-de13-4ff7-95e3-33931148365b admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:59.002592 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-189de99d-de13-4ff7-95e3-33931148365b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:59.021488 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189de99d-de13-4ff7-95e3-33931148365b admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:59.080607 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12eb7e8-0077-4018-bb74-039153baa786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:59.081236 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12eb7e8-0077-4018-bb74-039153baa786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:59.081545 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12eb7e8-0077-4018-bb74-039153baa786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:59.081818 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12eb7e8-0077-4018-bb74-039153baa786 admin admin] PATH_INFO: `/v3/projects/8607f205aafa4e849e534753fdf74cf4` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:59.082924 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12eb7e8-0077-4018-bb74-039153baa786 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8607f205aafa4e849e534753fdf74cf4)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:59.084812 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12eb7e8-0077-4018-bb74-039153baa786 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:59.131427 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-c12eb7e8-0077-4018-bb74-039153baa786 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8607f205aafa4e849e534753fdf74cf4', 'request_id': 'req-c12eb7e8-0077-4018-bb74-039153baa786'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:59.139360 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-c12eb7e8-0077-4018-bb74-039153baa786 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8607f205aafa4e849e534753fdf74cf4', 'request_id': 'req-c12eb7e8-0077-4018-bb74-039153baa786'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:59.145085 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-c12eb7e8-0077-4018-bb74-039153baa786 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8607f205aafa4e849e534753fdf74cf4', 'request_id': 'req-c12eb7e8-0077-4018-bb74-039153baa786'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:59.156859 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-c12eb7e8-0077-4018-bb74-039153baa786 admin admin] The token cache is being invalidate because project 8607f205aafa4e849e534753fdf74cf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:34:59.157151 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-c12eb7e8-0077-4018-bb74-039153baa786 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:59.158018 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-c12eb7e8-0077-4018-bb74-039153baa786 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:34:59.160100 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1326/2639] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:34:59 2022] DELETE /identity/v3/projects/8607f205aafa4e849e534753fdf74cf4 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:34:59.164876 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cefd30-fc0e-4041-b27c-fc6330568e07 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:59.165166 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cefd30-fc0e-4041-b27c-fc6330568e07 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:59.165470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cefd30-fc0e-4041-b27c-fc6330568e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:59.184934 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a5cefd30-fc0e-4041-b27c-fc6330568e07 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:34:59.186179 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a5cefd30-fc0e-4041-b27c-fc6330568e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:59.246620 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1314/2640] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:34:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:34:59.254251 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee5948-422d-4888-abe6-cbdb8a4b9117 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:59.254543 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee5948-422d-4888-abe6-cbdb8a4b9117 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:59.254826 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee5948-422d-4888-abe6-cbdb8a4b9117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:59.274580 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-75ee5948-422d-4888-abe6-cbdb8a4b9117 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:34:59.275780 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-75ee5948-422d-4888-abe6-cbdb8a4b9117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:59.284265 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1327/2641] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:34:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:34:59.288312 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cefd30-fc0e-4041-b27c-fc6330568e07 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:59.297838 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cefd30-fc0e-4041-b27c-fc6330568e07 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:59.302346 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83636fdc-b99d-4a9d-9f5e-0f7a1ed5ab18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:59.302930 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83636fdc-b99d-4a9d-9f5e-0f7a1ed5ab18 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:59.303207 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83636fdc-b99d-4a9d-9f5e-0f7a1ed5ab18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:59.303479 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83636fdc-b99d-4a9d-9f5e-0f7a1ed5ab18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:59.304115 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83636fdc-b99d-4a9d-9f5e-0f7a1ed5ab18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:34:59.304462 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83636fdc-b99d-4a9d-9f5e-0f7a1ed5ab18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:59.306247 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83636fdc-b99d-4a9d-9f5e-0f7a1ed5ab18 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:59.307726 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:34:59.321034 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1315/2642] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:34:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:34:59.326070 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee5948-422d-4888-abe6-cbdb8a4b9117 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:59.335609 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee5948-422d-4888-abe6-cbdb8a4b9117 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:59.341035 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77b9e9a-2ee0-4201-ae61-e4add9bc5ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:59.341661 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b9e9a-2ee0-4201-ae61-e4add9bc5ad7 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:59.341939 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b9e9a-2ee0-4201-ae61-e4add9bc5ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:59.342210 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b9e9a-2ee0-4201-ae61-e4add9bc5ad7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:59.342827 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77b9e9a-2ee0-4201-ae61-e4add9bc5ad7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:59.344990 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77b9e9a-2ee0-4201-ae61-e4add9bc5ad7 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:59.372836 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1328/2643] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:34:59 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:34:59.377150 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83636fdc-b99d-4a9d-9f5e-0f7a1ed5ab18 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:59.387798 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83636fdc-b99d-4a9d-9f5e-0f7a1ed5ab18 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:59.445052 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c6357-159e-46f7-b5da-0f1a262c3086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:59.445721 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c6357-159e-46f7-b5da-0f1a262c3086 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:59.446005 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c6357-159e-46f7-b5da-0f1a262c3086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:59.446280 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c6357-159e-46f7-b5da-0f1a262c3086 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:59.446910 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c6357-159e-46f7-b5da-0f1a262c3086 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:59.448814 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c6357-159e-46f7-b5da-0f1a262c3086 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:59.471264 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1316/2644] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:34:59 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:34:59.475218 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77b9e9a-2ee0-4201-ae61-e4add9bc5ad7 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:59.486466 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77b9e9a-2ee0-4201-ae61-e4add9bc5ad7 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:59.492519 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c739eb-fa62-46fa-a2db-6ec64cfecc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:59.493195 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c739eb-fa62-46fa-a2db-6ec64cfecc11 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:59.493573 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c739eb-fa62-46fa-a2db-6ec64cfecc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:59.493915 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c739eb-fa62-46fa-a2db-6ec64cfecc11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:59.494625 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c739eb-fa62-46fa-a2db-6ec64cfecc11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:59.496866 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c739eb-fa62-46fa-a2db-6ec64cfecc11 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:59.508643 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1329/2645] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:34:59.517085 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c6357-159e-46f7-b5da-0f1a262c3086 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:59.528447 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c6357-159e-46f7-b5da-0f1a262c3086 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:59.533135 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d860b9b-7823-441d-8867-975efd490e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:59.533769 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d860b9b-7823-441d-8867-975efd490e17 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:59.534078 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d860b9b-7823-441d-8867-975efd490e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:59.534376 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d860b9b-7823-441d-8867-975efd490e17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:59.535014 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d860b9b-7823-441d-8867-975efd490e17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:59.536591 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d860b9b-7823-441d-8867-975efd490e17 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:59.547464 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1317/2646] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:34:59.551831 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c739eb-fa62-46fa-a2db-6ec64cfecc11 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:59.562446 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c739eb-fa62-46fa-a2db-6ec64cfecc11 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:59.568085 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f2e00-f2cc-432e-beb7-59960ff27686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:59.568790 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f2e00-f2cc-432e-beb7-59960ff27686 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:59.569071 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f2e00-f2cc-432e-beb7-59960ff27686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:59.569393 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f2e00-f2cc-432e-beb7-59960ff27686 admin admin] PATH_INFO: `/v3/projects/64c241febeb1493d99e93d3dde6711ec/users/7c5cc81919f0421f90ea137fddd8e73d/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:59.585871 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f2e00-f2cc-432e-beb7-59960ff27686 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64c241febeb1493d99e93d3dde6711ec, user_id=7c5cc81919f0421f90ea137fddd8e73d, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:59.588584 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f2e00-f2cc-432e-beb7-59960ff27686 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:59.601413 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1330/2647] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:34:59 2022] PUT /identity/v3/projects/64c241febeb1493d99e93d3dde6711ec/users/7c5cc81919f0421f90ea137fddd8e73d/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:34:59.612200 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231d4c9-5132-4629-85a5-0099331eb546 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:59.612485 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231d4c9-5132-4629-85a5-0099331eb546 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:59.612778 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231d4c9-5132-4629-85a5-0099331eb546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:59.640579 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-8231d4c9-5132-4629-85a5-0099331eb546 None None] MFA Rules not processed for user `7c5cc81919f0421f90ea137fddd8e73d`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:34:59.642019 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8231d4c9-5132-4629-85a5-0099331eb546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:34:59.710321 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1318/2648] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:34:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:34:59.714901 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f2e00-f2cc-432e-beb7-59960ff27686 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:59.725768 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:34:59.782177 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f2e00-f2cc-432e-beb7-59960ff27686 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:59.787770 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef2921-c506-45d5-8dab-07e5efc602b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:59.788488 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef2921-c506-45d5-8dab-07e5efc602b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:59.788752 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef2921-c506-45d5-8dab-07e5efc602b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:59.789029 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef2921-c506-45d5-8dab-07e5efc602b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:59.789728 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ef2921-c506-45d5-8dab-07e5efc602b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:59.791763 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ef2921-c506-45d5-8dab-07e5efc602b8 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:59.825283 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1331/2649] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:34:59 2022] POST /identity/v3/projects => generated 387 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:34:59.826108 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231d4c9-5132-4629-85a5-0099331eb546 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:59.841517 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231d4c9-5132-4629-85a5-0099331eb546 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:59.905935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05012e1c-595e-4355-9154-fc0f0b3297be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:59.906577 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05012e1c-595e-4355-9154-fc0f0b3297be admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:59.906870 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05012e1c-595e-4355-9154-fc0f0b3297be admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:59.907139 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05012e1c-595e-4355-9154-fc0f0b3297be admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:59.907959 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05012e1c-595e-4355-9154-fc0f0b3297be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:59.909919 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05012e1c-595e-4355-9154-fc0f0b3297be admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:59.919732 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:34:59.931955 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1319/2650] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:34:59 2022] POST /identity/v3/users => generated 300 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:34:59.935884 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef2921-c506-45d5-8dab-07e5efc602b8 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:59.946106 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef2921-c506-45d5-8dab-07e5efc602b8 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:59.950727 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9897deac-3a8b-48de-b19a-7bd318a85e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:59.951319 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897deac-3a8b-48de-b19a-7bd318a85e2d admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:59.951597 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897deac-3a8b-48de-b19a-7bd318a85e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:59.951865 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897deac-3a8b-48de-b19a-7bd318a85e2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:34:59.952500 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9897deac-3a8b-48de-b19a-7bd318a85e2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:34:59.954122 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9897deac-3a8b-48de-b19a-7bd318a85e2d admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:34:59.964563 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1332/2651] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:34:59.968804 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05012e1c-595e-4355-9154-fc0f0b3297be admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:34:59.978672 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05012e1c-595e-4355-9154-fc0f0b3297be admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:34:59.983831 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a2245a-a765-4766-83de-34a9c2e23a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:34:59.984473 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2245a-a765-4766-83de-34a9c2e23a74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:34:59.984750 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2245a-a765-4766-83de-34a9c2e23a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:34:59.985025 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2245a-a765-4766-83de-34a9c2e23a74 admin admin] PATH_INFO: `/v3/projects/8b85bced0a6746b1af1098a8ecd0628d/users/c331b71f896d4cf78a5ad2e8a4e825ce/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:00.000275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a2245a-a765-4766-83de-34a9c2e23a74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b85bced0a6746b1af1098a8ecd0628d, user_id=c331b71f896d4cf78a5ad2e8a4e825ce, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:00.002981 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a2245a-a765-4766-83de-34a9c2e23a74 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:00.015401 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1320/2652] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:34:59 2022] PUT /identity/v3/projects/8b85bced0a6746b1af1098a8ecd0628d/users/c331b71f896d4cf78a5ad2e8a4e825ce/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:35:00.019274 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9897deac-3a8b-48de-b19a-7bd318a85e2d admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:00.029058 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9897deac-3a8b-48de-b19a-7bd318a85e2d admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:00.085236 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0c3cfc-a46b-46fc-9dda-c23d9005377e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:00.085867 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c3cfc-a46b-46fc-9dda-c23d9005377e admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:00.086148 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c3cfc-a46b-46fc-9dda-c23d9005377e admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:00.086423 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c3cfc-a46b-46fc-9dda-c23d9005377e admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:00.087070 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0c3cfc-a46b-46fc-9dda-c23d9005377e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:00.088665 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0c3cfc-a46b-46fc-9dda-c23d9005377e admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:00.098314 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1333/2653] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:35:00 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:35:00.102476 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a2245a-a765-4766-83de-34a9c2e23a74 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:00.114452 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a2245a-a765-4766-83de-34a9c2e23a74 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:00.119056 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa3a69-0d51-4213-af66-41f39a74076f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:00.119705 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa3a69-0d51-4213-af66-41f39a74076f admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:00.119984 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa3a69-0d51-4213-af66-41f39a74076f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:00.120256 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa3a69-0d51-4213-af66-41f39a74076f admin admin] PATH_INFO: `/v3/projects/8b85bced0a6746b1af1098a8ecd0628d/users/c331b71f896d4cf78a5ad2e8a4e825ce/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:00.122781 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa3a69-0d51-4213-af66-41f39a74076f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b85bced0a6746b1af1098a8ecd0628d, user_id=c331b71f896d4cf78a5ad2e8a4e825ce, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:00.125436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa3a69-0d51-4213-af66-41f39a74076f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:00.135730 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1321/2654] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:35:00 2022] PUT /identity/v3/projects/8b85bced0a6746b1af1098a8ecd0628d/users/c331b71f896d4cf78a5ad2e8a4e825ce/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:35:00.147044 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d4b95-eae8-4bd0-802f-2785059c8a9f None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:00.147573 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d4b95-eae8-4bd0-802f-2785059c8a9f None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:00.147687 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d4b95-eae8-4bd0-802f-2785059c8a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:00.175441 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-890d4b95-eae8-4bd0-802f-2785059c8a9f None None] MFA Rules not processed for user `c331b71f896d4cf78a5ad2e8a4e825ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:35:00.176636 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-890d4b95-eae8-4bd0-802f-2785059c8a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:00.243886 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1334/2655] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:35:00 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:35:00.255426 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa3a69-0d51-4213-af66-41f39a74076f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:00.257049 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-fafa3a69-0d51-4213-af66-41f39a74076f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:00.314052 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa3a69-0d51-4213-af66-41f39a74076f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:00.317714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819be83f-932d-4d92-9d33-5c1f74aaedd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:00.318398 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819be83f-932d-4d92-9d33-5c1f74aaedd5 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:00.318675 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819be83f-932d-4d92-9d33-5c1f74aaedd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:00.318946 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819be83f-932d-4d92-9d33-5c1f74aaedd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:00.328561 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819be83f-932d-4d92-9d33-5c1f74aaedd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:00.330013 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819be83f-932d-4d92-9d33-5c1f74aaedd5 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:00.339337 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1322/2656] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:35:00 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:35:01.035729 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d4b95-eae8-4bd0-802f-2785059c8a9f None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:01.040584 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:35:01.106217 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d4b95-eae8-4bd0-802f-2785059c8a9f None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:01.110979 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afdb3a2-b746-42f0-9fa7-928db0bd6e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:01.111613 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afdb3a2-b746-42f0-9fa7-928db0bd6e6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:01.111893 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afdb3a2-b746-42f0-9fa7-928db0bd6e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:01.112167 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afdb3a2-b746-42f0-9fa7-928db0bd6e6e admin admin] PATH_INFO: `/v3/users/7c5cc81919f0421f90ea137fddd8e73d` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:01.113355 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afdb3a2-b746-42f0-9fa7-928db0bd6e6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c5cc81919f0421f90ea137fddd8e73d)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:01.115267 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afdb3a2-b746-42f0-9fa7-928db0bd6e6e admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:01.165734 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-5afdb3a2-b746-42f0-9fa7-928db0bd6e6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c5cc81919f0421f90ea137fddd8e73d'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:01.177822 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-5afdb3a2-b746-42f0-9fa7-928db0bd6e6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c5cc81919f0421f90ea137fddd8e73d'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:01.185411 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-5afdb3a2-b746-42f0-9fa7-928db0bd6e6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c5cc81919f0421f90ea137fddd8e73d'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:01.192537 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-5afdb3a2-b746-42f0-9fa7-928db0bd6e6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c5cc81919f0421f90ea137fddd8e73d'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:01.193187 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-5afdb3a2-b746-42f0-9fa7-928db0bd6e6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c5cc81919f0421f90ea137fddd8e73d'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:01.200237 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1335/2657] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:35:01 2022] DELETE /identity/v3/users/7c5cc81919f0421f90ea137fddd8e73d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:35:01.205513 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec1159-37c4-446f-be04-c11629deab84 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:01.205807 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec1159-37c4-446f-be04-c11629deab84 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:01.206108 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec1159-37c4-446f-be04-c11629deab84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:01.224739 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-45ec1159-37c4-446f-be04-c11629deab84 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:35:01.225980 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-45ec1159-37c4-446f-be04-c11629deab84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:01.287706 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1323/2658] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:35:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:35:01.297494 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afdb3a2-b746-42f0-9fa7-928db0bd6e6e admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:01.298914 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-5afdb3a2-b746-42f0-9fa7-928db0bd6e6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:01.356802 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afdb3a2-b746-42f0-9fa7-928db0bd6e6e admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:01.360606 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce7385a-b950-47e3-a68b-edd41efc2d4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:01.361211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7385a-b950-47e3-a68b-edd41efc2d4e service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:01.361516 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7385a-b950-47e3-a68b-edd41efc2d4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:01.361793 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7385a-b950-47e3-a68b-edd41efc2d4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:01.370654 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce7385a-b950-47e3-a68b-edd41efc2d4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:01.372649 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce7385a-b950-47e3-a68b-edd41efc2d4e service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:01.381919 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1336/2659] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:35:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:35:01.646869 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec1159-37c4-446f-be04-c11629deab84 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:01.648385 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-45ec1159-37c4-446f-be04-c11629deab84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:01.660986 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec1159-37c4-446f-be04-c11629deab84 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:01.666319 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feed24c-5f1a-4819-acb5-640b8713328b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:01.667067 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feed24c-5f1a-4819-acb5-640b8713328b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:01.667353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feed24c-5f1a-4819-acb5-640b8713328b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:01.667632 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feed24c-5f1a-4819-acb5-640b8713328b admin admin] PATH_INFO: `/v3/projects/64c241febeb1493d99e93d3dde6711ec` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:01.669355 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feed24c-5f1a-4819-acb5-640b8713328b admin admin] RBAC: Authorizing `identity:delete_project(project_id=64c241febeb1493d99e93d3dde6711ec)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:01.671273 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feed24c-5f1a-4819-acb5-640b8713328b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:01.718160 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7feed24c-5f1a-4819-acb5-640b8713328b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64c241febeb1493d99e93d3dde6711ec', 'request_id': 'req-7feed24c-5f1a-4819-acb5-640b8713328b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:01.720018 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:INSERT=4,keystone:SELECT=91,keystone:DELETE=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:35:01.724689 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7feed24c-5f1a-4819-acb5-640b8713328b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64c241febeb1493d99e93d3dde6711ec', 'request_id': 'req-7feed24c-5f1a-4819-acb5-640b8713328b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:01.732004 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7feed24c-5f1a-4819-acb5-640b8713328b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64c241febeb1493d99e93d3dde6711ec', 'request_id': 'req-7feed24c-5f1a-4819-acb5-640b8713328b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:01.744420 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7feed24c-5f1a-4819-acb5-640b8713328b admin admin] The token cache is being invalidate because project 64c241febeb1493d99e93d3dde6711ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:35:01.744705 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7feed24c-5f1a-4819-acb5-640b8713328b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:01.745352 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7feed24c-5f1a-4819-acb5-640b8713328b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:01.746905 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1324/2660] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:35:01 2022] DELETE /identity/v3/projects/64c241febeb1493d99e93d3dde6711ec => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:35:01.750835 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce7385a-b950-47e3-a68b-edd41efc2d4e service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:01.760203 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce7385a-b950-47e3-a68b-edd41efc2d4e service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:01.782157 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:35:01.819335 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1aa538-3992-4485-a41d-8845c6bca841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:01.819979 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1aa538-3992-4485-a41d-8845c6bca841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:01.820239 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1aa538-3992-4485-a41d-8845c6bca841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:01.820508 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1aa538-3992-4485-a41d-8845c6bca841 admin admin] PATH_INFO: `/v3/users/c331b71f896d4cf78a5ad2e8a4e825ce` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:01.821640 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1aa538-3992-4485-a41d-8845c6bca841 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c331b71f896d4cf78a5ad2e8a4e825ce)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:01.823561 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1aa538-3992-4485-a41d-8845c6bca841 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:01.868337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-4e1aa538-3992-4485-a41d-8845c6bca841 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c331b71f896d4cf78a5ad2e8a4e825ce'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:01.880026 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-4e1aa538-3992-4485-a41d-8845c6bca841 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c331b71f896d4cf78a5ad2e8a4e825ce'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:01.886851 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-4e1aa538-3992-4485-a41d-8845c6bca841 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c331b71f896d4cf78a5ad2e8a4e825ce'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:01.894764 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-4e1aa538-3992-4485-a41d-8845c6bca841 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c331b71f896d4cf78a5ad2e8a4e825ce'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:01.895334 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-4e1aa538-3992-4485-a41d-8845c6bca841 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c331b71f896d4cf78a5ad2e8a4e825ce'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:01.901125 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1337/2661] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:35:01 2022] DELETE /identity/v3/users/c331b71f896d4cf78a5ad2e8a4e825ce => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:35:02.169560 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feed24c-5f1a-4819-acb5-640b8713328b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:02.171047 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7feed24c-5f1a-4819-acb5-640b8713328b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:02.184904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feed24c-5f1a-4819-acb5-640b8713328b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:02.255484 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da7b9a-b3b4-40a5-bdbe-cefec1ef76b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:02.256124 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da7b9a-b3b4-40a5-bdbe-cefec1ef76b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:02.256409 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da7b9a-b3b4-40a5-bdbe-cefec1ef76b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:02.256683 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da7b9a-b3b4-40a5-bdbe-cefec1ef76b6 admin admin] PATH_INFO: `/v3/projects/8b85bced0a6746b1af1098a8ecd0628d` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:02.257837 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29da7b9a-b3b4-40a5-bdbe-cefec1ef76b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b85bced0a6746b1af1098a8ecd0628d)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:02.259714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29da7b9a-b3b4-40a5-bdbe-cefec1ef76b6 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:02.305382 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-29da7b9a-b3b4-40a5-bdbe-cefec1ef76b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b85bced0a6746b1af1098a8ecd0628d', 'request_id': 'req-29da7b9a-b3b4-40a5-bdbe-cefec1ef76b6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:02.311116 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-29da7b9a-b3b4-40a5-bdbe-cefec1ef76b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b85bced0a6746b1af1098a8ecd0628d', 'request_id': 'req-29da7b9a-b3b4-40a5-bdbe-cefec1ef76b6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:02.318020 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-29da7b9a-b3b4-40a5-bdbe-cefec1ef76b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b85bced0a6746b1af1098a8ecd0628d', 'request_id': 'req-29da7b9a-b3b4-40a5-bdbe-cefec1ef76b6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:02.331410 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-29da7b9a-b3b4-40a5-bdbe-cefec1ef76b6 admin admin] The token cache is being invalidate because project 8b85bced0a6746b1af1098a8ecd0628d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:35:02.331693 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-29da7b9a-b3b4-40a5-bdbe-cefec1ef76b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:02.332330 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-29da7b9a-b3b4-40a5-bdbe-cefec1ef76b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:02.337537 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1325/2662] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:35:02 2022] DELETE /identity/v3/projects/8b85bced0a6746b1af1098a8ecd0628d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:35:02.339763 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c36f93-81c7-411c-8331-08e8e504313e None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:02.340774 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c36f93-81c7-411c-8331-08e8e504313e None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:02.340901 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c36f93-81c7-411c-8331-08e8e504313e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:02.362127 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-f4c36f93-81c7-411c-8331-08e8e504313e None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:35:02.363355 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-f4c36f93-81c7-411c-8331-08e8e504313e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:02.423572 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1338/2663] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:35:02.431368 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8e624-453b-476d-b027-cc6d785bce43 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:02.431654 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8e624-453b-476d-b027-cc6d785bce43 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:02.431930 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8e624-453b-476d-b027-cc6d785bce43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:02.453014 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-2aa8e624-453b-476d-b027-cc6d785bce43 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:35:02.454265 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2aa8e624-453b-476d-b027-cc6d785bce43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:02.462130 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1326/2664] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:35:02.466922 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c36f93-81c7-411c-8331-08e8e504313e None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:02.476418 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c36f93-81c7-411c-8331-08e8e504313e None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:02.481626 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c99dc-3e5b-421e-b952-4e0821d388b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:02.482232 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c99dc-3e5b-421e-b952-4e0821d388b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:02.482515 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c99dc-3e5b-421e-b952-4e0821d388b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:02.482786 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c99dc-3e5b-421e-b952-4e0821d388b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:02.483429 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193c99dc-3e5b-421e-b952-4e0821d388b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:35:02.483732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193c99dc-3e5b-421e-b952-4e0821d388b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:02.485616 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193c99dc-3e5b-421e-b952-4e0821d388b6 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:02.497330 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1339/2665] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:35:02.501870 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8e624-453b-476d-b027-cc6d785bce43 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:02.512031 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8e624-453b-476d-b027-cc6d785bce43 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:02.517387 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2c974-7933-4501-8ab8-1c08b6d2242c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:02.518297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2c974-7933-4501-8ab8-1c08b6d2242c admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:02.518449 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2c974-7933-4501-8ab8-1c08b6d2242c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:02.518610 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2c974-7933-4501-8ab8-1c08b6d2242c admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:02.519299 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b2c974-7933-4501-8ab8-1c08b6d2242c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:02.521372 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b2c974-7933-4501-8ab8-1c08b6d2242c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:02.549582 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1327/2666] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:35:02 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:35:02.554220 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c99dc-3e5b-421e-b952-4e0821d388b6 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:02.564462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c99dc-3e5b-421e-b952-4e0821d388b6 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:02.595721 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:35:02.625924 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a1d3e-2419-49fd-8000-6e91905cf53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:02.626524 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a1d3e-2419-49fd-8000-6e91905cf53f admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:02.626809 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a1d3e-2419-49fd-8000-6e91905cf53f admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:02.627091 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a1d3e-2419-49fd-8000-6e91905cf53f admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:02.627722 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a1d3e-2419-49fd-8000-6e91905cf53f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:02.629683 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a1d3e-2419-49fd-8000-6e91905cf53f admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:02.650512 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1340/2667] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:35:02 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:35:02.655275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2c974-7933-4501-8ab8-1c08b6d2242c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:02.665564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2c974-7933-4501-8ab8-1c08b6d2242c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:02.670779 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37629d2-1487-4fd1-a9cb-2966825ddf35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:02.671378 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37629d2-1487-4fd1-a9cb-2966825ddf35 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:02.671653 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37629d2-1487-4fd1-a9cb-2966825ddf35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:02.671920 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37629d2-1487-4fd1-a9cb-2966825ddf35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:02.672581 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37629d2-1487-4fd1-a9cb-2966825ddf35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:02.674176 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37629d2-1487-4fd1-a9cb-2966825ddf35 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:02.684013 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1328/2668] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:35:02.687955 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a1d3e-2419-49fd-8000-6e91905cf53f admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:02.697867 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a1d3e-2419-49fd-8000-6e91905cf53f admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:02.702441 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d5022-70bd-4e94-9024-d09745814564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:02.703033 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d5022-70bd-4e94-9024-d09745814564 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:02.703313 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d5022-70bd-4e94-9024-d09745814564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:02.703585 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d5022-70bd-4e94-9024-d09745814564 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:02.704254 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67d5022-70bd-4e94-9024-d09745814564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:02.705856 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67d5022-70bd-4e94-9024-d09745814564 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:02.716904 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1341/2669] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:35:02.721027 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37629d2-1487-4fd1-a9cb-2966825ddf35 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:02.732072 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37629d2-1487-4fd1-a9cb-2966825ddf35 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:02.736901 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f33409-cbb7-4240-b945-bb658f37a777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:02.737577 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f33409-cbb7-4240-b945-bb658f37a777 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:02.737904 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f33409-cbb7-4240-b945-bb658f37a777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:02.738297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f33409-cbb7-4240-b945-bb658f37a777 admin admin] PATH_INFO: `/v3/projects/946ec98211d54a399317b0499c2aa635/users/204ae1601971417991cfb9c84ab2ec43/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:02.752621 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f33409-cbb7-4240-b945-bb658f37a777 admin admin] RBAC: Authorizing `identity:create_grant(project_id=946ec98211d54a399317b0499c2aa635, user_id=204ae1601971417991cfb9c84ab2ec43, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:02.755371 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f33409-cbb7-4240-b945-bb658f37a777 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:02.768167 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1329/2670] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:35:02 2022] PUT /identity/v3/projects/946ec98211d54a399317b0499c2aa635/users/204ae1601971417991cfb9c84ab2ec43/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:35:02.773159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6713c-05de-4425-89b4-6b5298ba3a87 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:02.773542 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6713c-05de-4425-89b4-6b5298ba3a87 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:02.773824 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6713c-05de-4425-89b4-6b5298ba3a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:02.792685 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-aee6713c-05de-4425-89b4-6b5298ba3a87 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:35:02.794440 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-aee6713c-05de-4425-89b4-6b5298ba3a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:02.855270 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1342/2671] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:35:02.865110 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f33409-cbb7-4240-b945-bb658f37a777 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:02.866832 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-31f33409-cbb7-4240-b945-bb658f37a777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:02.876656 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:35:02.926781 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f33409-cbb7-4240-b945-bb658f37a777 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:02.930344 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b2fa0-59bb-48d7-8a28-7ecf5442dc26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:02.930941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b2fa0-59bb-48d7-8a28-7ecf5442dc26 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:02.931216 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b2fa0-59bb-48d7-8a28-7ecf5442dc26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:02.931486 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b2fa0-59bb-48d7-8a28-7ecf5442dc26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:02.940375 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0b2fa0-59bb-48d7-8a28-7ecf5442dc26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:02.941814 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0b2fa0-59bb-48d7-8a28-7ecf5442dc26 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:02.951498 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1330/2672] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:35:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:35:03.618850 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e70bc0-56b0-4725-bd4f-de22da1b735b None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:03.619410 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e70bc0-56b0-4725-bd4f-de22da1b735b None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:03.619521 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e70bc0-56b0-4725-bd4f-de22da1b735b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:03.648721 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-c1e70bc0-56b0-4725-bd4f-de22da1b735b None None] MFA Rules not processed for user `204ae1601971417991cfb9c84ab2ec43`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:35:03.650002 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-c1e70bc0-56b0-4725-bd4f-de22da1b735b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:03.722852 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1343/2673] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:35:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:35:03.727743 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b2fa0-59bb-48d7-8a28-7ecf5442dc26 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:03.738310 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b2fa0-59bb-48d7-8a28-7ecf5442dc26 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:03.742969 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff5cae-6058-4af4-a327-af3ca932e206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:03.743571 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff5cae-6058-4af4-a327-af3ca932e206 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:03.743847 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff5cae-6058-4af4-a327-af3ca932e206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:03.744117 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff5cae-6058-4af4-a327-af3ca932e206 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:03.744740 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ff5cae-6058-4af4-a327-af3ca932e206 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:03.746668 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ff5cae-6058-4af4-a327-af3ca932e206 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:03.775141 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1331/2674] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:35:03 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:35:03.779514 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e70bc0-56b0-4725-bd4f-de22da1b735b None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:03.790159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e70bc0-56b0-4725-bd4f-de22da1b735b None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:03.832773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:35:03.861690 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef0837-5860-4c0f-9b08-ef5e5b65f659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:03.862573 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef0837-5860-4c0f-9b08-ef5e5b65f659 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:03.863097 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef0837-5860-4c0f-9b08-ef5e5b65f659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:03.863561 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef0837-5860-4c0f-9b08-ef5e5b65f659 admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:03.864472 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef0837-5860-4c0f-9b08-ef5e5b65f659 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:03.868791 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef0837-5860-4c0f-9b08-ef5e5b65f659 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:03.895670 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1344/2675] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:35:03 2022] POST /identity/v3/users => generated 292 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:35:03.899999 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff5cae-6058-4af4-a327-af3ca932e206 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:03.912318 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff5cae-6058-4af4-a327-af3ca932e206 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:03.917366 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75572cd-6db9-4054-836a-bd224b55663b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:03.917937 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75572cd-6db9-4054-836a-bd224b55663b admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:03.918217 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75572cd-6db9-4054-836a-bd224b55663b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:03.918502 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75572cd-6db9-4054-836a-bd224b55663b admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:03.919134 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75572cd-6db9-4054-836a-bd224b55663b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:03.920718 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75572cd-6db9-4054-836a-bd224b55663b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:03.933609 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1332/2676] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:35:03.938086 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef0837-5860-4c0f-9b08-ef5e5b65f659 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:03.949886 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef0837-5860-4c0f-9b08-ef5e5b65f659 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:03.957276 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310cec20-5c19-46fd-a12f-7b190cbba497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:03.957276 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310cec20-5c19-46fd-a12f-7b190cbba497 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:03.957276 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310cec20-5c19-46fd-a12f-7b190cbba497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:03.957276 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310cec20-5c19-46fd-a12f-7b190cbba497 admin admin] PATH_INFO: `/v3/projects/00bc77badaf24b708b71ba175dfc1a6f/users/d206cc7c8ef14ec4ab751c6f7c358f22/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:03.972527 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310cec20-5c19-46fd-a12f-7b190cbba497 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00bc77badaf24b708b71ba175dfc1a6f, user_id=d206cc7c8ef14ec4ab751c6f7c358f22, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:03.975605 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310cec20-5c19-46fd-a12f-7b190cbba497 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:03.991289 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1345/2677] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:35:03 2022] PUT /identity/v3/projects/00bc77badaf24b708b71ba175dfc1a6f/users/d206cc7c8ef14ec4ab751c6f7c358f22/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:35:03.995464 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75572cd-6db9-4054-836a-bd224b55663b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:04.008873 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75572cd-6db9-4054-836a-bd224b55663b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:04.065805 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798968fe-22b2-4d9e-8f42-780ee085e243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:04.066419 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798968fe-22b2-4d9e-8f42-780ee085e243 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:04.066701 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798968fe-22b2-4d9e-8f42-780ee085e243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:04.066976 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798968fe-22b2-4d9e-8f42-780ee085e243 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:04.067612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798968fe-22b2-4d9e-8f42-780ee085e243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:04.069193 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798968fe-22b2-4d9e-8f42-780ee085e243 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:04.079678 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1333/2678] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:35:04.083958 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310cec20-5c19-46fd-a12f-7b190cbba497 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:04.094860 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310cec20-5c19-46fd-a12f-7b190cbba497 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:04.099651 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dfcd76-8f55-4f58-85b1-02676c872914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:04.100422 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfcd76-8f55-4f58-85b1-02676c872914 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:04.100822 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfcd76-8f55-4f58-85b1-02676c872914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:04.101196 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfcd76-8f55-4f58-85b1-02676c872914 admin admin] PATH_INFO: `/v3/projects/00bc77badaf24b708b71ba175dfc1a6f/users/d206cc7c8ef14ec4ab751c6f7c358f22/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:04.103236 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dfcd76-8f55-4f58-85b1-02676c872914 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00bc77badaf24b708b71ba175dfc1a6f, user_id=d206cc7c8ef14ec4ab751c6f7c358f22, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:04.106038 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dfcd76-8f55-4f58-85b1-02676c872914 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:04.117645 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1346/2679] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:35:04 2022] PUT /identity/v3/projects/00bc77badaf24b708b71ba175dfc1a6f/users/d206cc7c8ef14ec4ab751c6f7c358f22/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:35:04.438992 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798968fe-22b2-4d9e-8f42-780ee085e243 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:04.441712 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-798968fe-22b2-4d9e-8f42-780ee085e243 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:04.456559 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798968fe-22b2-4d9e-8f42-780ee085e243 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:04.522607 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520cccb2-2eb8-4821-9235-d04fb114c648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:04.523422 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520cccb2-2eb8-4821-9235-d04fb114c648 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:04.523772 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520cccb2-2eb8-4821-9235-d04fb114c648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:04.524116 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520cccb2-2eb8-4821-9235-d04fb114c648 admin admin] PATH_INFO: `/v3/users/7176d1895248400887578343dfef6a77` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:04.525370 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520cccb2-2eb8-4821-9235-d04fb114c648 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7176d1895248400887578343dfef6a77)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:04.527320 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520cccb2-2eb8-4821-9235-d04fb114c648 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:04.544750 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:35:04.580106 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-520cccb2-2eb8-4821-9235-d04fb114c648 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7176d1895248400887578343dfef6a77'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:04.587546 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-520cccb2-2eb8-4821-9235-d04fb114c648 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7176d1895248400887578343dfef6a77'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:04.599873 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-520cccb2-2eb8-4821-9235-d04fb114c648 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7176d1895248400887578343dfef6a77'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:04.607570 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-520cccb2-2eb8-4821-9235-d04fb114c648 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7176d1895248400887578343dfef6a77'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:04.608197 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-520cccb2-2eb8-4821-9235-d04fb114c648 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7176d1895248400887578343dfef6a77'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:04.614587 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1334/2680] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:35:04 2022] DELETE /identity/v3/users/7176d1895248400887578343dfef6a77 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:35:04.619336 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a212978-a565-4f29-bb38-ab7e8f08cc21 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:04.619819 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a212978-a565-4f29-bb38-ab7e8f08cc21 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:04.619999 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a212978-a565-4f29-bb38-ab7e8f08cc21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:04.639941 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-1a212978-a565-4f29-bb38-ab7e8f08cc21 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:35:04.641238 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-1a212978-a565-4f29-bb38-ab7e8f08cc21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:04.706483 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1347/2681] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:35:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:35:04.716962 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520cccb2-2eb8-4821-9235-d04fb114c648 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:04.718494 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-520cccb2-2eb8-4821-9235-d04fb114c648 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:04.755232 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35e45b-91e3-4f06-af8f-c4d4964b9187 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:04.755684 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35e45b-91e3-4f06-af8f-c4d4964b9187 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:04.756102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35e45b-91e3-4f06-af8f-c4d4964b9187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:04.794890 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-0d35e45b-91e3-4f06-af8f-c4d4964b9187 None None] MFA Rules not processed for user `d206cc7c8ef14ec4ab751c6f7c358f22`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:35:04.796124 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-0d35e45b-91e3-4f06-af8f-c4d4964b9187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:04.807451 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520cccb2-2eb8-4821-9235-d04fb114c648 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:04.812647 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb1d5c-49a1-4422-951d-b401614780fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:04.813377 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb1d5c-49a1-4422-951d-b401614780fd service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:04.813576 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb1d5c-49a1-4422-951d-b401614780fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:04.813868 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb1d5c-49a1-4422-951d-b401614780fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:04.827700 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceb1d5c-49a1-4422-951d-b401614780fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:04.828846 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceb1d5c-49a1-4422-951d-b401614780fd service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:04.840072 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1335/2682] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:35:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:35:04.883055 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1348/2683] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:35:04 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:35:04.895744 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb1d5c-49a1-4422-951d-b401614780fd service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:04.898250 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4ceb1d5c-49a1-4422-951d-b401614780fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:04.969271 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb1d5c-49a1-4422-951d-b401614780fd service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:04.973036 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17618352-ee0f-41c4-84c0-4a89e889116b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:04.974000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17618352-ee0f-41c4-84c0-4a89e889116b service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:04.974000 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17618352-ee0f-41c4-84c0-4a89e889116b service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:04.974392 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17618352-ee0f-41c4-84c0-4a89e889116b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:04.976257 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-17618352-ee0f-41c4-84c0-4a89e889116b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:05.026948 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:35:05.041644 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17618352-ee0f-41c4-84c0-4a89e889116b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:05.043102 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17618352-ee0f-41c4-84c0-4a89e889116b service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:05.055794 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1336/2684] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:35:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:35:05.130873 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35e45b-91e3-4f06-af8f-c4d4964b9187 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:05.132378 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-0d35e45b-91e3-4f06-af8f-c4d4964b9187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:05.146156 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35e45b-91e3-4f06-af8f-c4d4964b9187 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:05.151075 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c1b6e-e23b-4f6a-95c9-e6e4f57f395a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:05.152415 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c1b6e-e23b-4f6a-95c9-e6e4f57f395a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:05.152706 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c1b6e-e23b-4f6a-95c9-e6e4f57f395a admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:05.152979 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c1b6e-e23b-4f6a-95c9-e6e4f57f395a admin admin] PATH_INFO: `/v3/projects/9d37e9b0f6e6404882c0c7d4b25fd04b` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:05.154184 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690c1b6e-e23b-4f6a-95c9-e6e4f57f395a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d37e9b0f6e6404882c0c7d4b25fd04b)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:05.156060 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690c1b6e-e23b-4f6a-95c9-e6e4f57f395a admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:05.168989 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:35:05.211306 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-690c1b6e-e23b-4f6a-95c9-e6e4f57f395a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d37e9b0f6e6404882c0c7d4b25fd04b', 'request_id': 'req-690c1b6e-e23b-4f6a-95c9-e6e4f57f395a'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:05.219871 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-690c1b6e-e23b-4f6a-95c9-e6e4f57f395a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d37e9b0f6e6404882c0c7d4b25fd04b', 'request_id': 'req-690c1b6e-e23b-4f6a-95c9-e6e4f57f395a'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:05.228769 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-690c1b6e-e23b-4f6a-95c9-e6e4f57f395a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d37e9b0f6e6404882c0c7d4b25fd04b', 'request_id': 'req-690c1b6e-e23b-4f6a-95c9-e6e4f57f395a'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:05.242241 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-690c1b6e-e23b-4f6a-95c9-e6e4f57f395a admin admin] The token cache is being invalidate because project 9d37e9b0f6e6404882c0c7d4b25fd04b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:35:05.242534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-690c1b6e-e23b-4f6a-95c9-e6e4f57f395a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:05.243187 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-690c1b6e-e23b-4f6a-95c9-e6e4f57f395a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:05.244873 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1349/2685] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:35:05 2022] DELETE /identity/v3/projects/9d37e9b0f6e6404882c0c7d4b25fd04b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:35:05.250646 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1216fc-d6dc-4771-8cef-fdc60b158c96 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:05.250938 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1216fc-d6dc-4771-8cef-fdc60b158c96 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:05.251218 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1216fc-d6dc-4771-8cef-fdc60b158c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:05.271283 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-9f1216fc-d6dc-4771-8cef-fdc60b158c96 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:35:05.272495 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9f1216fc-d6dc-4771-8cef-fdc60b158c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:05.336034 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1337/2686] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:35:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:35:05.344926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7333b55-b7a4-49da-98e2-fe3f9986e37b None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:05.345211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7333b55-b7a4-49da-98e2-fe3f9986e37b None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:05.345511 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7333b55-b7a4-49da-98e2-fe3f9986e37b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:05.367089 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-d7333b55-b7a4-49da-98e2-fe3f9986e37b None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:35:05.368317 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-d7333b55-b7a4-49da-98e2-fe3f9986e37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:05.381059 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1350/2687] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:35:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:35:05.388971 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1216fc-d6dc-4771-8cef-fdc60b158c96 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:05.399081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1216fc-d6dc-4771-8cef-fdc60b158c96 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:05.404899 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a3a3ba-379f-4d3d-8789-4671ca2b4075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:05.405540 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3a3ba-379f-4d3d-8789-4671ca2b4075 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:05.405822 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3a3ba-379f-4d3d-8789-4671ca2b4075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:05.406094 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3a3ba-379f-4d3d-8789-4671ca2b4075 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:05.406743 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a3a3ba-379f-4d3d-8789-4671ca2b4075 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:35:05.407048 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a3a3ba-379f-4d3d-8789-4671ca2b4075 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:05.408688 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a3a3ba-379f-4d3d-8789-4671ca2b4075 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:05.421915 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1338/2688] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:35:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:35:05.426581 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7333b55-b7a4-49da-98e2-fe3f9986e37b None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:05.437145 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7333b55-b7a4-49da-98e2-fe3f9986e37b None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:05.441789 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b7d77-b448-474c-b12a-873ffce95af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:05.442379 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b7d77-b448-474c-b12a-873ffce95af3 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:05.442656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b7d77-b448-474c-b12a-873ffce95af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:05.442922 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b7d77-b448-474c-b12a-873ffce95af3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:05.443532 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8b7d77-b448-474c-b12a-873ffce95af3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:05.445446 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8b7d77-b448-474c-b12a-873ffce95af3 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:05.472843 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1351/2689] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:35:05 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:35:05.477035 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a3a3ba-379f-4d3d-8789-4671ca2b4075 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:05.487818 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a3a3ba-379f-4d3d-8789-4671ca2b4075 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:05.544307 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f46a54-18ac-42cd-afed-09a138264855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:05.544843 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f46a54-18ac-42cd-afed-09a138264855 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:05.545129 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f46a54-18ac-42cd-afed-09a138264855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:05.545446 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f46a54-18ac-42cd-afed-09a138264855 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:05.546083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f46a54-18ac-42cd-afed-09a138264855 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:05.548051 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f46a54-18ac-42cd-afed-09a138264855 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:05.568296 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1339/2690] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:35:05 2022] POST /identity/v3/users => generated 296 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:35:05.572721 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b7d77-b448-474c-b12a-873ffce95af3 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:05.582902 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b7d77-b448-474c-b12a-873ffce95af3 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:05.587511 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7683f-8e84-4083-976a-0bd5b87b7563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:05.588091 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7683f-8e84-4083-976a-0bd5b87b7563 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:05.588365 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7683f-8e84-4083-976a-0bd5b87b7563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:05.588634 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7683f-8e84-4083-976a-0bd5b87b7563 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:05.589280 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d7683f-8e84-4083-976a-0bd5b87b7563 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:05.591399 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d7683f-8e84-4083-976a-0bd5b87b7563 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:05.601232 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1352/2691] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:35:05.605275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f46a54-18ac-42cd-afed-09a138264855 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:05.615980 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f46a54-18ac-42cd-afed-09a138264855 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:05.620640 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc8bda-187d-491d-90cd-8d77706d4246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:05.621782 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc8bda-187d-491d-90cd-8d77706d4246 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:05.622067 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc8bda-187d-491d-90cd-8d77706d4246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:05.622343 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc8bda-187d-491d-90cd-8d77706d4246 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:05.623136 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fc8bda-187d-491d-90cd-8d77706d4246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:05.624646 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fc8bda-187d-491d-90cd-8d77706d4246 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:05.634937 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1340/2692] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:35:05.639458 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7683f-8e84-4083-976a-0bd5b87b7563 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:05.649387 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7683f-8e84-4083-976a-0bd5b87b7563 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:05.654524 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb837b5e-119a-4944-ba52-84ca5c007a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:05.655187 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb837b5e-119a-4944-ba52-84ca5c007a9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:05.655470 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb837b5e-119a-4944-ba52-84ca5c007a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:05.655749 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb837b5e-119a-4944-ba52-84ca5c007a9b admin admin] PATH_INFO: `/v3/projects/9442165dc944494a96784dcdeb006fa1/users/3043d53ac2cc4580be7cec2be4d8e6ee/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:05.670769 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb837b5e-119a-4944-ba52-84ca5c007a9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9442165dc944494a96784dcdeb006fa1, user_id=3043d53ac2cc4580be7cec2be4d8e6ee, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:05.673462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb837b5e-119a-4944-ba52-84ca5c007a9b admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:05.687114 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1353/2693] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:35:05 2022] PUT /identity/v3/projects/9442165dc944494a96784dcdeb006fa1/users/3043d53ac2cc4580be7cec2be4d8e6ee/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:35:05.697375 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f233d-c0b7-4227-8dcd-d344067782fa None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:05.697659 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f233d-c0b7-4227-8dcd-d344067782fa None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:05.697931 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f233d-c0b7-4227-8dcd-d344067782fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:05.726919 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-686f233d-c0b7-4227-8dcd-d344067782fa None None] MFA Rules not processed for user `3043d53ac2cc4580be7cec2be4d8e6ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:35:05.728119 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-686f233d-c0b7-4227-8dcd-d344067782fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:05.770184 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:35:05.791305 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1341/2694] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:35:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:35:05.802718 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb837b5e-119a-4944-ba52-84ca5c007a9b admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:05.804411 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-cb837b5e-119a-4944-ba52-84ca5c007a9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:05.860715 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb837b5e-119a-4944-ba52-84ca5c007a9b admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:05.864267 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3f1e4-49e3-4637-83a8-032c5d0d990c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:05.864862 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3f1e4-49e3-4637-83a8-032c5d0d990c service cinder] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:05.865141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3f1e4-49e3-4637-83a8-032c5d0d990c service cinder] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:05.865442 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3f1e4-49e3-4637-83a8-032c5d0d990c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:05.873623 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b3f1e4-49e3-4637-83a8-032c5d0d990c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:05.875041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b3f1e4-49e3-4637-83a8-032c5d0d990c service cinder] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:05.883692 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1354/2695] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:35:05 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:35:06.457323 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686f233d-c0b7-4227-8dcd-d344067782fa None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:06.458803 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-686f233d-c0b7-4227-8dcd-d344067782fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:06.471044 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686f233d-c0b7-4227-8dcd-d344067782fa None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:06.474520 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752b4655-a8a1-472a-a65d-b80ddf275433 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:06.475111 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b4655-a8a1-472a-a65d-b80ddf275433 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:06.475387 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b4655-a8a1-472a-a65d-b80ddf275433 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:06.475661 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b4655-a8a1-472a-a65d-b80ddf275433 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:06.477308 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-752b4655-a8a1-472a-a65d-b80ddf275433 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:06.538702 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752b4655-a8a1-472a-a65d-b80ddf275433 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:06.540174 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752b4655-a8a1-472a-a65d-b80ddf275433 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:06.550111 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1342/2696] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:35:06 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:35:15.879794 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=72,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:35:16.545832 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=37 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:35:24.312700 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3f1e4-49e3-4637-83a8-032c5d0d990c service cinder] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:24.376380 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3f1e4-49e3-4637-83a8-032c5d0d990c service cinder] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:24.381059 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa786fb-1291-49eb-9027-1c4734982d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:24.381719 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa786fb-1291-49eb-9027-1c4734982d83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:24.382001 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa786fb-1291-49eb-9027-1c4734982d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:24.382304 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa786fb-1291-49eb-9027-1c4734982d83 admin admin] PATH_INFO: `/v3/users/3043d53ac2cc4580be7cec2be4d8e6ee` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:24.383635 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa786fb-1291-49eb-9027-1c4734982d83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3043d53ac2cc4580be7cec2be4d8e6ee)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:24.385564 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa786fb-1291-49eb-9027-1c4734982d83 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:24.433102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-daa786fb-1291-49eb-9027-1c4734982d83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3043d53ac2cc4580be7cec2be4d8e6ee'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:24.445466 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-daa786fb-1291-49eb-9027-1c4734982d83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3043d53ac2cc4580be7cec2be4d8e6ee'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:24.452791 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-daa786fb-1291-49eb-9027-1c4734982d83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3043d53ac2cc4580be7cec2be4d8e6ee'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:24.460743 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-daa786fb-1291-49eb-9027-1c4734982d83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3043d53ac2cc4580be7cec2be4d8e6ee'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:24.461414 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-daa786fb-1291-49eb-9027-1c4734982d83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3043d53ac2cc4580be7cec2be4d8e6ee'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:24.468177 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1355/2697] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:35:24 2022] DELETE /identity/v3/users/3043d53ac2cc4580be7cec2be4d8e6ee => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:35:24.472647 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e4a8a-3629-4b85-a384-e026d3a05f31 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:24.473145 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e4a8a-3629-4b85-a384-e026d3a05f31 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:24.473261 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e4a8a-3629-4b85-a384-e026d3a05f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:24.492187 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f38e4a8a-3629-4b85-a384-e026d3a05f31 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:35:24.493434 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f38e4a8a-3629-4b85-a384-e026d3a05f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:24.561187 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1343/2698] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:35:24.570792 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa786fb-1291-49eb-9027-1c4734982d83 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:24.572307 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-daa786fb-1291-49eb-9027-1c4734982d83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:24.631782 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa786fb-1291-49eb-9027-1c4734982d83 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:24.635626 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd61a4-0317-4342-b704-4d6f5b4cce78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:24.636265 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd61a4-0317-4342-b704-4d6f5b4cce78 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:24.636547 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd61a4-0317-4342-b704-4d6f5b4cce78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:24.636819 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd61a4-0317-4342-b704-4d6f5b4cce78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:24.646162 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd61a4-0317-4342-b704-4d6f5b4cce78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:24.647712 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd61a4-0317-4342-b704-4d6f5b4cce78 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:24.661803 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1356/2699] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:35:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:35:25.038363 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38e4a8a-3629-4b85-a384-e026d3a05f31 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:25.040364 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f38e4a8a-3629-4b85-a384-e026d3a05f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:25.066768 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38e4a8a-3629-4b85-a384-e026d3a05f31 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:25.073828 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098959b1-c770-4653-a5f6-6f929e31f8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:25.074747 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098959b1-c770-4653-a5f6-6f929e31f8e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:25.075182 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098959b1-c770-4653-a5f6-6f929e31f8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:25.075314 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098959b1-c770-4653-a5f6-6f929e31f8e3 admin admin] PATH_INFO: `/v3/projects/9442165dc944494a96784dcdeb006fa1` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:25.076810 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098959b1-c770-4653-a5f6-6f929e31f8e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9442165dc944494a96784dcdeb006fa1)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:25.079232 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098959b1-c770-4653-a5f6-6f929e31f8e3 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:25.159894 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-098959b1-c770-4653-a5f6-6f929e31f8e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9442165dc944494a96784dcdeb006fa1', 'request_id': 'req-098959b1-c770-4653-a5f6-6f929e31f8e3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:25.168425 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-098959b1-c770-4653-a5f6-6f929e31f8e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9442165dc944494a96784dcdeb006fa1', 'request_id': 'req-098959b1-c770-4653-a5f6-6f929e31f8e3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:25.175670 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-098959b1-c770-4653-a5f6-6f929e31f8e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9442165dc944494a96784dcdeb006fa1', 'request_id': 'req-098959b1-c770-4653-a5f6-6f929e31f8e3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:25.191067 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-098959b1-c770-4653-a5f6-6f929e31f8e3 admin admin] The token cache is being invalidate because project 9442165dc944494a96784dcdeb006fa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:35:25.191418 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-098959b1-c770-4653-a5f6-6f929e31f8e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:25.192696 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-098959b1-c770-4653-a5f6-6f929e31f8e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:25.194478 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1344/2700] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:35:25 2022] DELETE /identity/v3/projects/9442165dc944494a96784dcdeb006fa1 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:35:25.200543 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65b4c9-cb4d-4797-b7ca-43301748c433 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:25.201090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65b4c9-cb4d-4797-b7ca-43301748c433 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:25.201761 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65b4c9-cb4d-4797-b7ca-43301748c433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:25.224517 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-eb65b4c9-cb4d-4797-b7ca-43301748c433 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:35:25.225573 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-eb65b4c9-cb4d-4797-b7ca-43301748c433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:25.244842 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:35:25.295263 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1357/2701] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:35:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:35:25.303580 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344e884-c57b-4d4f-9a92-f5f9e1a19e69 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:25.303863 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344e884-c57b-4d4f-9a92-f5f9e1a19e69 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:25.304138 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344e884-c57b-4d4f-9a92-f5f9e1a19e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:25.329276 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-3344e884-c57b-4d4f-9a92-f5f9e1a19e69 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:35:25.331309 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3344e884-c57b-4d4f-9a92-f5f9e1a19e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:25.343936 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1345/2702] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:35:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:35:25.348733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65b4c9-cb4d-4797-b7ca-43301748c433 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:25.362826 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65b4c9-cb4d-4797-b7ca-43301748c433 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:25.374278 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e658f5-597b-45f8-becb-55afa20eaed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:25.375096 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e658f5-597b-45f8-becb-55afa20eaed6 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:25.375661 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e658f5-597b-45f8-becb-55afa20eaed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:25.376018 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e658f5-597b-45f8-becb-55afa20eaed6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:25.376809 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e658f5-597b-45f8-becb-55afa20eaed6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:35:25.377105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e658f5-597b-45f8-becb-55afa20eaed6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:25.379330 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e658f5-597b-45f8-becb-55afa20eaed6 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:25.395090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1358/2703] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:35:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:35:25.399579 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344e884-c57b-4d4f-9a92-f5f9e1a19e69 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:25.414161 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344e884-c57b-4d4f-9a92-f5f9e1a19e69 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:25.426226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973694c6-cf18-47fd-98cc-5299ec5c5485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:25.427192 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973694c6-cf18-47fd-98cc-5299ec5c5485 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:25.427696 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973694c6-cf18-47fd-98cc-5299ec5c5485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:25.428187 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973694c6-cf18-47fd-98cc-5299ec5c5485 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:25.429023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973694c6-cf18-47fd-98cc-5299ec5c5485 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:25.431338 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973694c6-cf18-47fd-98cc-5299ec5c5485 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:25.469312 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1346/2704] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:35:25 2022] POST /identity/v3/projects => generated 379 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:35:25.473462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e658f5-597b-45f8-becb-55afa20eaed6 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:25.488171 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e658f5-597b-45f8-becb-55afa20eaed6 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:25.556864 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d358f-868d-40f9-b504-91701c968add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:25.557934 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d358f-868d-40f9-b504-91701c968add admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:25.557934 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d358f-868d-40f9-b504-91701c968add admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:25.558865 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d358f-868d-40f9-b504-91701c968add admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:25.559581 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d358f-868d-40f9-b504-91701c968add admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:25.561804 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d358f-868d-40f9-b504-91701c968add admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:25.587389 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1359/2705] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:35:25 2022] POST /identity/v3/users => generated 290 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:35:25.591578 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973694c6-cf18-47fd-98cc-5299ec5c5485 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:25.604162 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973694c6-cf18-47fd-98cc-5299ec5c5485 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:25.612574 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1308441-f5f0-4944-9f47-45c6711355a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:25.613435 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1308441-f5f0-4944-9f47-45c6711355a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:25.613773 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1308441-f5f0-4944-9f47-45c6711355a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:25.614244 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1308441-f5f0-4944-9f47-45c6711355a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:25.615037 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1308441-f5f0-4944-9f47-45c6711355a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:25.616914 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1308441-f5f0-4944-9f47-45c6711355a2 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:25.630766 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1347/2706] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:35:25.633289 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d358f-868d-40f9-b504-91701c968add admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:25.646299 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d358f-868d-40f9-b504-91701c968add admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:25.652698 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d402a-b7d7-450d-bfe5-c4580695ad1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:25.653578 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d402a-b7d7-450d-bfe5-c4580695ad1a admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:25.653934 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d402a-b7d7-450d-bfe5-c4580695ad1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:25.654281 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d402a-b7d7-450d-bfe5-c4580695ad1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:25.655005 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d402a-b7d7-450d-bfe5-c4580695ad1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:25.660518 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d402a-b7d7-450d-bfe5-c4580695ad1a admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:25.680823 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1360/2707] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:35:25.685377 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1308441-f5f0-4944-9f47-45c6711355a2 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:25.696722 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1308441-f5f0-4944-9f47-45c6711355a2 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:25.702624 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077d29f3-2d36-4da7-b483-71e2e944075d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:25.703883 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d29f3-2d36-4da7-b483-71e2e944075d admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:25.704243 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d29f3-2d36-4da7-b483-71e2e944075d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:25.704586 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d29f3-2d36-4da7-b483-71e2e944075d admin admin] PATH_INFO: `/v3/projects/7e12f0c37bd14176b04dfd87d7b10520/users/63371d392c8c4b73939ee73a1627c8dd/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:25.722715 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077d29f3-2d36-4da7-b483-71e2e944075d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e12f0c37bd14176b04dfd87d7b10520, user_id=63371d392c8c4b73939ee73a1627c8dd, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:25.725845 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077d29f3-2d36-4da7-b483-71e2e944075d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:25.747495 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1348/2708] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:35:25 2022] PUT /identity/v3/projects/7e12f0c37bd14176b04dfd87d7b10520/users/63371d392c8c4b73939ee73a1627c8dd/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:35:25.752315 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6ed96-6304-440d-811f-ecc25205b498 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:25.752600 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6ed96-6304-440d-811f-ecc25205b498 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:25.752872 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6ed96-6304-440d-811f-ecc25205b498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:25.772188 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-7ba6ed96-6304-440d-811f-ecc25205b498 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:35:25.773684 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7ba6ed96-6304-440d-811f-ecc25205b498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:25.851121 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1361/2709] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:35:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:35:25.862469 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077d29f3-2d36-4da7-b483-71e2e944075d admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:25.864270 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-077d29f3-2d36-4da7-b483-71e2e944075d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:25.893295 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:35:25.945697 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077d29f3-2d36-4da7-b483-71e2e944075d admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:25.951725 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283643c-c4fb-4341-9444-fcf7faa0fe2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:25.952536 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283643c-c4fb-4341-9444-fcf7faa0fe2b service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:25.952825 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283643c-c4fb-4341-9444-fcf7faa0fe2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:25.954131 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283643c-c4fb-4341-9444-fcf7faa0fe2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:25.965666 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283643c-c4fb-4341-9444-fcf7faa0fe2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:25.967006 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283643c-c4fb-4341-9444-fcf7faa0fe2b service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:25.981032 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1349/2710] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:35:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:35:26.602667 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6ed96-6304-440d-811f-ecc25205b498 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:26.605562 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7ba6ed96-6304-440d-811f-ecc25205b498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:26.624124 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6ed96-6304-440d-811f-ecc25205b498 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:26.628702 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffad551-a456-4932-b5d1-5480d627c051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:26.629579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffad551-a456-4932-b5d1-5480d627c051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:26.629860 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffad551-a456-4932-b5d1-5480d627c051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:26.630133 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffad551-a456-4932-b5d1-5480d627c051 admin admin] PATH_INFO: `/v3/users/204ae1601971417991cfb9c84ab2ec43` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:26.631282 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffad551-a456-4932-b5d1-5480d627c051 admin admin] RBAC: Authorizing `identity:delete_user(user_id=204ae1601971417991cfb9c84ab2ec43)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:26.633217 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffad551-a456-4932-b5d1-5480d627c051 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:26.664599 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:35:26.690175 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-bffad551-a456-4932-b5d1-5480d627c051 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '204ae1601971417991cfb9c84ab2ec43'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:26.703043 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-bffad551-a456-4932-b5d1-5480d627c051 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '204ae1601971417991cfb9c84ab2ec43'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:26.709926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-bffad551-a456-4932-b5d1-5480d627c051 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '204ae1601971417991cfb9c84ab2ec43'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:26.718404 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-bffad551-a456-4932-b5d1-5480d627c051 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '204ae1601971417991cfb9c84ab2ec43'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:26.719174 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-bffad551-a456-4932-b5d1-5480d627c051 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '204ae1601971417991cfb9c84ab2ec43'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:26.726733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1362/2711] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:35:26 2022] DELETE /identity/v3/users/204ae1601971417991cfb9c84ab2ec43 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:35:26.902655 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283643c-c4fb-4341-9444-fcf7faa0fe2b service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:26.914259 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283643c-c4fb-4341-9444-fcf7faa0fe2b service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:26.997896 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4b02e-f7a6-4c77-ba89-8f048f769149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:26.998660 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4b02e-f7a6-4c77-ba89-8f048f769149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:26.999549 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4b02e-f7a6-4c77-ba89-8f048f769149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:26.999821 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4b02e-f7a6-4c77-ba89-8f048f769149 admin admin] PATH_INFO: `/v3/projects/946ec98211d54a399317b0499c2aa635` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:27.002504 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced4b02e-f7a6-4c77-ba89-8f048f769149 admin admin] RBAC: Authorizing `identity:delete_project(project_id=946ec98211d54a399317b0499c2aa635)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:27.003513 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced4b02e-f7a6-4c77-ba89-8f048f769149 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:27.053581 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-ced4b02e-f7a6-4c77-ba89-8f048f769149 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '946ec98211d54a399317b0499c2aa635', 'request_id': 'req-ced4b02e-f7a6-4c77-ba89-8f048f769149'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:27.059591 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-ced4b02e-f7a6-4c77-ba89-8f048f769149 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '946ec98211d54a399317b0499c2aa635', 'request_id': 'req-ced4b02e-f7a6-4c77-ba89-8f048f769149'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:27.066753 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-ced4b02e-f7a6-4c77-ba89-8f048f769149 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '946ec98211d54a399317b0499c2aa635', 'request_id': 'req-ced4b02e-f7a6-4c77-ba89-8f048f769149'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:27.081235 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-ced4b02e-f7a6-4c77-ba89-8f048f769149 admin admin] The token cache is being invalidate because project 946ec98211d54a399317b0499c2aa635 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:35:27.081575 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-ced4b02e-f7a6-4c77-ba89-8f048f769149 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:27.082225 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-ced4b02e-f7a6-4c77-ba89-8f048f769149 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:27.083965 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1350/2712] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:35:26 2022] DELETE /identity/v3/projects/946ec98211d54a399317b0499c2aa635 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:35:27.087679 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffad551-a456-4932-b5d1-5480d627c051 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:27.089487 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-bffad551-a456-4932-b5d1-5480d627c051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:27.103106 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffad551-a456-4932-b5d1-5480d627c051 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:27.163550 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb00bcb-0613-4bcd-b246-ca4fc06551f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:27.164498 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb00bcb-0613-4bcd-b246-ca4fc06551f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:27.164849 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb00bcb-0613-4bcd-b246-ca4fc06551f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:27.165183 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb00bcb-0613-4bcd-b246-ca4fc06551f6 admin admin] PATH_INFO: `/v3/users/d206cc7c8ef14ec4ab751c6f7c358f22` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:27.167439 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb00bcb-0613-4bcd-b246-ca4fc06551f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d206cc7c8ef14ec4ab751c6f7c358f22)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:27.169458 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb00bcb-0613-4bcd-b246-ca4fc06551f6 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:27.218475 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-2fb00bcb-0613-4bcd-b246-ca4fc06551f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd206cc7c8ef14ec4ab751c6f7c358f22'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:27.238654 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-2fb00bcb-0613-4bcd-b246-ca4fc06551f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd206cc7c8ef14ec4ab751c6f7c358f22'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:27.247105 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-2fb00bcb-0613-4bcd-b246-ca4fc06551f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd206cc7c8ef14ec4ab751c6f7c358f22'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:27.255232 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-2fb00bcb-0613-4bcd-b246-ca4fc06551f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd206cc7c8ef14ec4ab751c6f7c358f22'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:27.256193 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-2fb00bcb-0613-4bcd-b246-ca4fc06551f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd206cc7c8ef14ec4ab751c6f7c358f22'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:27.262674 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1363/2713] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:35:27 2022] DELETE /identity/v3/users/d206cc7c8ef14ec4ab751c6f7c358f22 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:35:27.466403 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4b02e-f7a6-4c77-ba89-8f048f769149 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:27.469303 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ced4b02e-f7a6-4c77-ba89-8f048f769149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:27.496705 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4b02e-f7a6-4c77-ba89-8f048f769149 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:27.571666 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:35:27.592205 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feeab46-8dfb-4090-a8e7-ac631632f6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:27.592877 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feeab46-8dfb-4090-a8e7-ac631632f6b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:27.593161 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feeab46-8dfb-4090-a8e7-ac631632f6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:27.593464 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feeab46-8dfb-4090-a8e7-ac631632f6b1 admin admin] PATH_INFO: `/v3/projects/00bc77badaf24b708b71ba175dfc1a6f` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:27.599291 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feeab46-8dfb-4090-a8e7-ac631632f6b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00bc77badaf24b708b71ba175dfc1a6f)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:27.600178 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feeab46-8dfb-4090-a8e7-ac631632f6b1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:27.658311 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6feeab46-8dfb-4090-a8e7-ac631632f6b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00bc77badaf24b708b71ba175dfc1a6f', 'request_id': 'req-6feeab46-8dfb-4090-a8e7-ac631632f6b1'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:27.665987 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6feeab46-8dfb-4090-a8e7-ac631632f6b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00bc77badaf24b708b71ba175dfc1a6f', 'request_id': 'req-6feeab46-8dfb-4090-a8e7-ac631632f6b1'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:27.684199 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6feeab46-8dfb-4090-a8e7-ac631632f6b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00bc77badaf24b708b71ba175dfc1a6f', 'request_id': 'req-6feeab46-8dfb-4090-a8e7-ac631632f6b1'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:27.701914 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6feeab46-8dfb-4090-a8e7-ac631632f6b1 admin admin] The token cache is being invalidate because project 00bc77badaf24b708b71ba175dfc1a6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:35:27.702337 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6feeab46-8dfb-4090-a8e7-ac631632f6b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:27.703013 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6feeab46-8dfb-4090-a8e7-ac631632f6b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:27.708149 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1351/2714] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:35:27 2022] DELETE /identity/v3/projects/00bc77badaf24b708b71ba175dfc1a6f => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:35:27.710076 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e974d4f-7233-4811-97b7-85abc6f35e40 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:27.710547 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e974d4f-7233-4811-97b7-85abc6f35e40 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:27.710633 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e974d4f-7233-4811-97b7-85abc6f35e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:27.736519 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-7e974d4f-7233-4811-97b7-85abc6f35e40 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:35:27.738407 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7e974d4f-7233-4811-97b7-85abc6f35e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:27.754178 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:35:27.826777 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1364/2715] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:35:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:35:27.828945 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6910c1a4-d85f-44dd-b5f7-1b7f2d6839b7 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:27.829235 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6910c1a4-d85f-44dd-b5f7-1b7f2d6839b7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:27.829547 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6910c1a4-d85f-44dd-b5f7-1b7f2d6839b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:27.856155 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-6910c1a4-d85f-44dd-b5f7-1b7f2d6839b7 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:35:27.857596 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6910c1a4-d85f-44dd-b5f7-1b7f2d6839b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:27.869893 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1352/2716] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:35:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:35:27.871127 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e974d4f-7233-4811-97b7-85abc6f35e40 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:27.882894 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e974d4f-7233-4811-97b7-85abc6f35e40 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:27.889557 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc2e0a-3482-4de8-b292-6c892b98f198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:27.890192 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc2e0a-3482-4de8-b292-6c892b98f198 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:27.890473 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc2e0a-3482-4de8-b292-6c892b98f198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:27.890921 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc2e0a-3482-4de8-b292-6c892b98f198 admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:27.891596 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc2e0a-3482-4de8-b292-6c892b98f198 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:35:27.891905 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc2e0a-3482-4de8-b292-6c892b98f198 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:27.894394 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc2e0a-3482-4de8-b292-6c892b98f198 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:27.911739 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1365/2717] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:35:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:35:27.913260 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6910c1a4-d85f-44dd-b5f7-1b7f2d6839b7 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:27.924191 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6910c1a4-d85f-44dd-b5f7-1b7f2d6839b7 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:27.930902 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d2fae-7fad-42c6-a128-12ec6289e04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:27.931534 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d2fae-7fad-42c6-a128-12ec6289e04e admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:27.931817 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d2fae-7fad-42c6-a128-12ec6289e04e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:27.932089 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d2fae-7fad-42c6-a128-12ec6289e04e admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:27.933701 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d2fae-7fad-42c6-a128-12ec6289e04e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:27.935753 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d2fae-7fad-42c6-a128-12ec6289e04e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:27.972664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1353/2718] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:35:27 2022] POST /identity/v3/projects => generated 359 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:35:27.976705 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc2e0a-3482-4de8-b292-6c892b98f198 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:27.993926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc2e0a-3482-4de8-b292-6c892b98f198 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:28.090033 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21876c9b-eebf-46c3-9f72-0793c6b83fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:28.091395 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21876c9b-eebf-46c3-9f72-0793c6b83fa5 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:28.091785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21876c9b-eebf-46c3-9f72-0793c6b83fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:28.092166 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21876c9b-eebf-46c3-9f72-0793c6b83fa5 admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:28.092906 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21876c9b-eebf-46c3-9f72-0793c6b83fa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:28.096403 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21876c9b-eebf-46c3-9f72-0793c6b83fa5 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:28.131805 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1366/2719] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:35:27 2022] POST /identity/v3/users => generated 280 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:35:28.135436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d2fae-7fad-42c6-a128-12ec6289e04e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:28.153268 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d2fae-7fad-42c6-a128-12ec6289e04e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:28.160739 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ec751-1fc5-4dad-b38b-6a2a3ed218cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:28.162073 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ec751-1fc5-4dad-b38b-6a2a3ed218cc admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:28.162424 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ec751-1fc5-4dad-b38b-6a2a3ed218cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:28.162877 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ec751-1fc5-4dad-b38b-6a2a3ed218cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:28.163612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33ec751-1fc5-4dad-b38b-6a2a3ed218cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:28.165626 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33ec751-1fc5-4dad-b38b-6a2a3ed218cc admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:28.181421 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1354/2720] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:35:28.186724 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21876c9b-eebf-46c3-9f72-0793c6b83fa5 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:28.197617 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21876c9b-eebf-46c3-9f72-0793c6b83fa5 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:28.203776 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86866d2-1a70-4fe1-9a48-3b73d554e143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:28.204512 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86866d2-1a70-4fe1-9a48-3b73d554e143 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:28.205026 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86866d2-1a70-4fe1-9a48-3b73d554e143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:28.205441 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86866d2-1a70-4fe1-9a48-3b73d554e143 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:28.206298 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86866d2-1a70-4fe1-9a48-3b73d554e143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:28.208003 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86866d2-1a70-4fe1-9a48-3b73d554e143 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:28.219968 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1367/2721] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:35:28.223925 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ec751-1fc5-4dad-b38b-6a2a3ed218cc admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:28.234541 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ec751-1fc5-4dad-b38b-6a2a3ed218cc admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:28.240618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380c0e6-842a-421c-96db-daf85d4cc205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:28.241381 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380c0e6-842a-421c-96db-daf85d4cc205 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:28.241670 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380c0e6-842a-421c-96db-daf85d4cc205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:28.241961 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380c0e6-842a-421c-96db-daf85d4cc205 admin admin] PATH_INFO: `/v3/projects/fe5f7f8bff8946d4abd55b83b788df2e/users/0635b19d20a841cfb19070addc970847/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:28.260996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380c0e6-842a-421c-96db-daf85d4cc205 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe5f7f8bff8946d4abd55b83b788df2e, user_id=0635b19d20a841cfb19070addc970847, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:28.264390 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380c0e6-842a-421c-96db-daf85d4cc205 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:28.279898 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1355/2722] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:35:28 2022] PUT /identity/v3/projects/fe5f7f8bff8946d4abd55b83b788df2e/users/0635b19d20a841cfb19070addc970847/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:35:28.293017 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e01b57-b020-4828-b5e3-52d5dfe3e443 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:28.293537 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e01b57-b020-4828-b5e3-52d5dfe3e443 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:28.294007 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e01b57-b020-4828-b5e3-52d5dfe3e443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:28.366783 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-29e01b57-b020-4828-b5e3-52d5dfe3e443 None None] MFA Rules not processed for user `0635b19d20a841cfb19070addc970847`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:35:28.368345 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-29e01b57-b020-4828-b5e3-52d5dfe3e443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:28.468137 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1368/2723] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:35:28 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:35:28.512332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380c0e6-842a-421c-96db-daf85d4cc205 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:28.514430 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4380c0e6-842a-421c-96db-daf85d4cc205 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:28.643181 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380c0e6-842a-421c-96db-daf85d4cc205 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:28.650465 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa989f32-8cbb-46b0-9fca-adfd6440c229 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:28.651758 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa989f32-8cbb-46b0-9fca-adfd6440c229 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:28.652113 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa989f32-8cbb-46b0-9fca-adfd6440c229 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:28.652465 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa989f32-8cbb-46b0-9fca-adfd6440c229 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:28.668063 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa989f32-8cbb-46b0-9fca-adfd6440c229 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:28.669666 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa989f32-8cbb-46b0-9fca-adfd6440c229 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:28.680615 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1356/2724] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:35:28 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:35:28.699659 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e01b57-b020-4828-b5e3-52d5dfe3e443 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:28.730262 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:35:28.797034 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e01b57-b020-4828-b5e3-52d5dfe3e443 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:28.804302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320ab9a-402a-44e1-b163-a02a538d9ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:28.804950 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320ab9a-402a-44e1-b163-a02a538d9ddc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:28.805227 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320ab9a-402a-44e1-b163-a02a538d9ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:28.805524 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320ab9a-402a-44e1-b163-a02a538d9ddc admin admin] PATH_INFO: `/v3/users/0635b19d20a841cfb19070addc970847` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:28.807151 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2320ab9a-402a-44e1-b163-a02a538d9ddc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0635b19d20a841cfb19070addc970847)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:28.810244 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2320ab9a-402a-44e1-b163-a02a538d9ddc admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:28.872529 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-2320ab9a-402a-44e1-b163-a02a538d9ddc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0635b19d20a841cfb19070addc970847'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:28.887655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-2320ab9a-402a-44e1-b163-a02a538d9ddc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0635b19d20a841cfb19070addc970847'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:28.894992 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-2320ab9a-402a-44e1-b163-a02a538d9ddc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0635b19d20a841cfb19070addc970847'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:28.905719 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-2320ab9a-402a-44e1-b163-a02a538d9ddc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0635b19d20a841cfb19070addc970847'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:28.907107 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-2320ab9a-402a-44e1-b163-a02a538d9ddc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0635b19d20a841cfb19070addc970847'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:28.918147 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1369/2725] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:35:28 2022] DELETE /identity/v3/users/0635b19d20a841cfb19070addc970847 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:35:28.923542 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4ca8b-109c-4e70-bb8a-b1d5143ba442 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:28.924080 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4ca8b-109c-4e70-bb8a-b1d5143ba442 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:28.924329 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4ca8b-109c-4e70-bb8a-b1d5143ba442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:28.947351 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-49e4ca8b-109c-4e70-bb8a-b1d5143ba442 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:35:28.949900 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-49e4ca8b-109c-4e70-bb8a-b1d5143ba442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:28.969516 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:35:29.031769 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1357/2726] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:35:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:35:29.040908 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320ab9a-402a-44e1-b163-a02a538d9ddc admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:29.044406 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-2320ab9a-402a-44e1-b163-a02a538d9ddc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:29.114728 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27f27b-db15-45fc-9d39-618e08aededf None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:29.115019 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27f27b-db15-45fc-9d39-618e08aededf None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:29.115296 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27f27b-db15-45fc-9d39-618e08aededf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:29.128056 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320ab9a-402a-44e1-b163-a02a538d9ddc admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:29.132168 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80180c8d-5c8a-49ff-969e-d6ded4cc44e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:29.132872 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80180c8d-5c8a-49ff-969e-d6ded4cc44e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:29.133214 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80180c8d-5c8a-49ff-969e-d6ded4cc44e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:29.134255 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80180c8d-5c8a-49ff-969e-d6ded4cc44e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:29.145951 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80180c8d-5c8a-49ff-969e-d6ded4cc44e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:29.153265 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80180c8d-5c8a-49ff-969e-d6ded4cc44e6 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:29.159337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1370/2727] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:35:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 20:35:29.164764 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-4b27f27b-db15-45fc-9d39-618e08aededf None None] MFA Rules not processed for user `f0eca860c013457ea79e4633f3e6d895`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:35:29.166316 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4b27f27b-db15-45fc-9d39-618e08aededf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:29.246242 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1358/2728] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:35:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:35:29.793447 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80180c8d-5c8a-49ff-969e-d6ded4cc44e6 service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:29.795825 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-80180c8d-5c8a-49ff-969e-d6ded4cc44e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:29.814925 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80180c8d-5c8a-49ff-969e-d6ded4cc44e6 service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:29.821573 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f016f0f4-2d57-4026-94f2-11b5c7e197e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:29.822532 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016f0f4-2d57-4026-94f2-11b5c7e197e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:29.830067 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016f0f4-2d57-4026-94f2-11b5c7e197e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:29.830740 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016f0f4-2d57-4026-94f2-11b5c7e197e4 admin admin] PATH_INFO: `/v3/projects/fe5f7f8bff8946d4abd55b83b788df2e` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:29.833378 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f016f0f4-2d57-4026-94f2-11b5c7e197e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe5f7f8bff8946d4abd55b83b788df2e)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:29.842328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f016f0f4-2d57-4026-94f2-11b5c7e197e4 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:29.898360 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:35:29.924897 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-f016f0f4-2d57-4026-94f2-11b5c7e197e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe5f7f8bff8946d4abd55b83b788df2e', 'request_id': 'req-f016f0f4-2d57-4026-94f2-11b5c7e197e4'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:29.936090 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-f016f0f4-2d57-4026-94f2-11b5c7e197e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe5f7f8bff8946d4abd55b83b788df2e', 'request_id': 'req-f016f0f4-2d57-4026-94f2-11b5c7e197e4'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:29.945091 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-f016f0f4-2d57-4026-94f2-11b5c7e197e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe5f7f8bff8946d4abd55b83b788df2e', 'request_id': 'req-f016f0f4-2d57-4026-94f2-11b5c7e197e4'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:29.963147 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-f016f0f4-2d57-4026-94f2-11b5c7e197e4 admin admin] The token cache is being invalidate because project fe5f7f8bff8946d4abd55b83b788df2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:35:29.964299 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-f016f0f4-2d57-4026-94f2-11b5c7e197e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:29.965692 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-f016f0f4-2d57-4026-94f2-11b5c7e197e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:29.967708 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1371/2729] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:35:29 2022] DELETE /identity/v3/projects/fe5f7f8bff8946d4abd55b83b788df2e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:35:29.974544 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8edda-dbb4-4421-b298-195f19acc54f None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:29.975089 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8edda-dbb4-4421-b298-195f19acc54f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:29.975620 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8edda-dbb4-4421-b298-195f19acc54f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:30.006359 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f8a8edda-dbb4-4421-b298-195f19acc54f None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:35:30.007821 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f8a8edda-dbb4-4421-b298-195f19acc54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:30.092690 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1359/2730] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:35:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:35:30.101655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8109d8-30c9-49d9-9130-a49e837e37a8 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:30.102419 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8109d8-30c9-49d9-9130-a49e837e37a8 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:30.103038 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8109d8-30c9-49d9-9130-a49e837e37a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:30.134802 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-cc8109d8-30c9-49d9-9130-a49e837e37a8 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:35:30.153284 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-cc8109d8-30c9-49d9-9130-a49e837e37a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:30.179156 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1372/2731] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:35:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:35:30.185649 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8edda-dbb4-4421-b298-195f19acc54f None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:30.201679 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8edda-dbb4-4421-b298-195f19acc54f None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:30.209036 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1544d11-62b5-4698-887d-4358b9703cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:30.210080 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1544d11-62b5-4698-887d-4358b9703cad admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:30.210536 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1544d11-62b5-4698-887d-4358b9703cad admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:30.210942 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1544d11-62b5-4698-887d-4358b9703cad admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:30.212927 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1544d11-62b5-4698-887d-4358b9703cad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:35:30.217079 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1544d11-62b5-4698-887d-4358b9703cad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:30.223483 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1544d11-62b5-4698-887d-4358b9703cad admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:30.271142 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1360/2732] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:35:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:35:30.272404 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8109d8-30c9-49d9-9130-a49e837e37a8 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:30.303463 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8109d8-30c9-49d9-9130-a49e837e37a8 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:30.344593 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa24338-d93b-4c1b-acdc-fa8daf709af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:30.345414 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa24338-d93b-4c1b-acdc-fa8daf709af6 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:30.345791 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa24338-d93b-4c1b-acdc-fa8daf709af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:30.346164 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa24338-d93b-4c1b-acdc-fa8daf709af6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:30.347801 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa24338-d93b-4c1b-acdc-fa8daf709af6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:30.357650 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa24338-d93b-4c1b-acdc-fa8daf709af6 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:30.411019 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1373/2733] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:35:30 2022] POST /identity/v3/projects => generated 371 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:35:30.415084 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1544d11-62b5-4698-887d-4358b9703cad admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:30.429365 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1544d11-62b5-4698-887d-4358b9703cad admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:30.522720 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:35:30.532535 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5071049f-78a0-4f8b-bb59-beff422ab90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:30.533226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071049f-78a0-4f8b-bb59-beff422ab90c admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:30.533684 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071049f-78a0-4f8b-bb59-beff422ab90c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:30.533989 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071049f-78a0-4f8b-bb59-beff422ab90c admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:30.534648 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5071049f-78a0-4f8b-bb59-beff422ab90c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:30.536760 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5071049f-78a0-4f8b-bb59-beff422ab90c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:30.564438 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1361/2734] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:35:30 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:35:30.569390 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa24338-d93b-4c1b-acdc-fa8daf709af6 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:30.583229 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa24338-d93b-4c1b-acdc-fa8daf709af6 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:30.589871 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed7663-fe8f-482d-b7b0-2847b03a6932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:30.590789 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed7663-fe8f-482d-b7b0-2847b03a6932 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:30.591299 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed7663-fe8f-482d-b7b0-2847b03a6932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:30.591407 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed7663-fe8f-482d-b7b0-2847b03a6932 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:30.592106 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed7663-fe8f-482d-b7b0-2847b03a6932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:30.594234 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed7663-fe8f-482d-b7b0-2847b03a6932 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:30.612555 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1374/2735] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:35:30.612862 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5071049f-78a0-4f8b-bb59-beff422ab90c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:30.628273 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5071049f-78a0-4f8b-bb59-beff422ab90c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:30.634005 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d3580-cb62-49f9-9716-498192977636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:30.634818 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d3580-cb62-49f9-9716-498192977636 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:30.636362 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d3580-cb62-49f9-9716-498192977636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:30.636686 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d3580-cb62-49f9-9716-498192977636 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:30.639254 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d3580-cb62-49f9-9716-498192977636 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:30.640853 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d3580-cb62-49f9-9716-498192977636 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:30.661147 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1362/2736] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:35:30.662298 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed7663-fe8f-482d-b7b0-2847b03a6932 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:30.676959 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed7663-fe8f-482d-b7b0-2847b03a6932 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:30.681990 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69695d1-d6a9-486f-a48f-a03b792635b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:30.682657 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69695d1-d6a9-486f-a48f-a03b792635b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:30.682937 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69695d1-d6a9-486f-a48f-a03b792635b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:30.683235 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69695d1-d6a9-486f-a48f-a03b792635b8 admin admin] PATH_INFO: `/v3/projects/4d04d8b1ba794ad6be2ecee6fbdcadf3/users/7b4826601da34342b2870b87e5ce8d9c/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:30.710115 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69695d1-d6a9-486f-a48f-a03b792635b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d04d8b1ba794ad6be2ecee6fbdcadf3, user_id=7b4826601da34342b2870b87e5ce8d9c, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:30.712968 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69695d1-d6a9-486f-a48f-a03b792635b8 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:30.734199 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1375/2737] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:35:30 2022] PUT /identity/v3/projects/4d04d8b1ba794ad6be2ecee6fbdcadf3/users/7b4826601da34342b2870b87e5ce8d9c/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:35:30.744692 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0784dbd-1c4e-4901-a553-f976251b90d4 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:30.745657 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0784dbd-1c4e-4901-a553-f976251b90d4 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:30.746011 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0784dbd-1c4e-4901-a553-f976251b90d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:30.776730 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a0784dbd-1c4e-4901-a553-f976251b90d4 None None] MFA Rules not processed for user `7b4826601da34342b2870b87e5ce8d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:35:30.778102 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a0784dbd-1c4e-4901-a553-f976251b90d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:30.873891 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1363/2738] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:35:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:35:30.887915 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69695d1-d6a9-486f-a48f-a03b792635b8 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:30.891460 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-e69695d1-d6a9-486f-a48f-a03b792635b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:30.970911 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69695d1-d6a9-486f-a48f-a03b792635b8 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:30.978640 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894220b-05d9-4f6e-9d43-b7c01bad5551 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5298486310004b63b51116dbc747e31d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:30.985595 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894220b-05d9-4f6e-9d43-b7c01bad5551 service glance] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:30.986100 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894220b-05d9-4f6e-9d43-b7c01bad5551 service glance] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:30.986609 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894220b-05d9-4f6e-9d43-b7c01bad5551 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:30.999523 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a894220b-05d9-4f6e-9d43-b7c01bad5551 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:31.002217 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a894220b-05d9-4f6e-9d43-b7c01bad5551 service glance] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:31.016224 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1376/2739] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:35:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:35:31.520565 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c865f-23dc-4887-9336-be0d4d60ac7d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:31.521224 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c865f-23dc-4887-9336-be0d4d60ac7d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:31.521224 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c865f-23dc-4887-9336-be0d4d60ac7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:31.553911 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-fd7c865f-23dc-4887-9336-be0d4d60ac7d None None] MFA Rules not processed for user `63371d392c8c4b73939ee73a1627c8dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:35:31.555085 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-fd7c865f-23dc-4887-9336-be0d4d60ac7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:31.737501 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1364/2740] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:35:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:35:31.742075 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894220b-05d9-4f6e-9d43-b7c01bad5551 service glance] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:31.743710 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a894220b-05d9-4f6e-9d43-b7c01bad5551 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:31.896972 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894220b-05d9-4f6e-9d43-b7c01bad5551 service glance] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:31.910302 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee6fd4-7a6e-43f7-80b4-83932e379d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:31.911057 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee6fd4-7a6e-43f7-80b4-83932e379d81 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:31.911443 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee6fd4-7a6e-43f7-80b4-83932e379d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:31.911828 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee6fd4-7a6e-43f7-80b4-83932e379d81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:31.912584 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee6fd4-7a6e-43f7-80b4-83932e379d81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:31.914937 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee6fd4-7a6e-43f7-80b4-83932e379d81 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:31.921464 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:35:31.959835 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1377/2741] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:35:31 2022] POST /identity/v3/projects => generated 379 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:35:31.964061 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c865f-23dc-4887-9336-be0d4d60ac7d None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:31.978902 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c865f-23dc-4887-9336-be0d4d60ac7d None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:32.051032 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:35:32.062854 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba000a00-e2e3-4c73-b059-29f8ec077814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:32.063601 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba000a00-e2e3-4c73-b059-29f8ec077814 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:32.063963 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba000a00-e2e3-4c73-b059-29f8ec077814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:32.064311 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba000a00-e2e3-4c73-b059-29f8ec077814 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:32.065167 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba000a00-e2e3-4c73-b059-29f8ec077814 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:32.067419 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba000a00-e2e3-4c73-b059-29f8ec077814 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:32.094530 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1365/2742] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:35:31 2022] POST /identity/v3/users => generated 296 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:35:32.098535 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee6fd4-7a6e-43f7-80b4-83932e379d81 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:32.112972 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee6fd4-7a6e-43f7-80b4-83932e379d81 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:32.121266 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b238e6f0-7cc6-4b91-9e39-378c3edaf9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:32.122121 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b238e6f0-7cc6-4b91-9e39-378c3edaf9e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:32.122464 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b238e6f0-7cc6-4b91-9e39-378c3edaf9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:32.122836 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b238e6f0-7cc6-4b91-9e39-378c3edaf9e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:32.123651 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b238e6f0-7cc6-4b91-9e39-378c3edaf9e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:32.125513 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b238e6f0-7cc6-4b91-9e39-378c3edaf9e3 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:32.148666 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1378/2743] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:35:32.153289 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba000a00-e2e3-4c73-b059-29f8ec077814 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:32.165160 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba000a00-e2e3-4c73-b059-29f8ec077814 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:32.176882 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea358b4-9df0-44d8-9184-b8d6f5c92545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:32.177832 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea358b4-9df0-44d8-9184-b8d6f5c92545 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:32.178236 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea358b4-9df0-44d8-9184-b8d6f5c92545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:32.178808 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea358b4-9df0-44d8-9184-b8d6f5c92545 admin admin] PATH_INFO: `/v3/projects/d8e762a2344248dc9e79ff464c26dddc/users/7114fe9707834ac1a3b03b4b41c3b36a/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:32.202512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea358b4-9df0-44d8-9184-b8d6f5c92545 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8e762a2344248dc9e79ff464c26dddc, user_id=7114fe9707834ac1a3b03b4b41c3b36a, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:32.206699 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea358b4-9df0-44d8-9184-b8d6f5c92545 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:32.224833 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1366/2744] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:35:32 2022] PUT /identity/v3/projects/d8e762a2344248dc9e79ff464c26dddc/users/7114fe9707834ac1a3b03b4b41c3b36a/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:35:32.228653 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b238e6f0-7cc6-4b91-9e39-378c3edaf9e3 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:32.242208 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b238e6f0-7cc6-4b91-9e39-378c3edaf9e3 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:32.313026 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08912a95-0ed1-4323-ae87-eebd16eb6abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:32.313701 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08912a95-0ed1-4323-ae87-eebd16eb6abf admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:32.313978 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08912a95-0ed1-4323-ae87-eebd16eb6abf admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:32.314247 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08912a95-0ed1-4323-ae87-eebd16eb6abf admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:32.314896 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08912a95-0ed1-4323-ae87-eebd16eb6abf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:32.316491 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08912a95-0ed1-4323-ae87-eebd16eb6abf admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:32.331303 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1379/2745] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:35:32.332985 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea358b4-9df0-44d8-9184-b8d6f5c92545 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:32.348421 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea358b4-9df0-44d8-9184-b8d6f5c92545 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:32.355320 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf790e-a4d5-47d3-b74e-ed9b4481eae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:32.356003 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf790e-a4d5-47d3-b74e-ed9b4481eae3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:32.356283 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf790e-a4d5-47d3-b74e-ed9b4481eae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:32.356558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf790e-a4d5-47d3-b74e-ed9b4481eae3 admin admin] PATH_INFO: `/v3/projects/d8e762a2344248dc9e79ff464c26dddc/users/7114fe9707834ac1a3b03b4b41c3b36a/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:32.359089 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbf790e-a4d5-47d3-b74e-ed9b4481eae3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8e762a2344248dc9e79ff464c26dddc, user_id=7114fe9707834ac1a3b03b4b41c3b36a, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:32.366324 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbf790e-a4d5-47d3-b74e-ed9b4481eae3 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:32.376402 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1367/2746] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:35:32 2022] PUT /identity/v3/projects/d8e762a2344248dc9e79ff464c26dddc/users/7114fe9707834ac1a3b03b4b41c3b36a/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:35:37.740632 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedeacc8-212c-44bc-8a2f-f942b4776a4b None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:37.741173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedeacc8-212c-44bc-8a2f-f942b4776a4b None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:37.741173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedeacc8-212c-44bc-8a2f-f942b4776a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:37.770344 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-cedeacc8-212c-44bc-8a2f-f942b4776a4b None None] MFA Rules not processed for user `7114fe9707834ac1a3b03b4b41c3b36a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:35:37.771923 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-cedeacc8-212c-44bc-8a2f-f942b4776a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:37.852439 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1380/2747] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:35:37 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:35:37.867637 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf790e-a4d5-47d3-b74e-ed9b4481eae3 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:37.870817 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2dbf790e-a4d5-47d3-b74e-ed9b4481eae3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:37.954042 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf790e-a4d5-47d3-b74e-ed9b4481eae3 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:37.959512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0180017-e65e-4c1c-b741-2aa0be1c64d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:37.960125 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0180017-e65e-4c1c-b741-2aa0be1c64d0 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:37.960402 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0180017-e65e-4c1c-b741-2aa0be1c64d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:37.960672 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0180017-e65e-4c1c-b741-2aa0be1c64d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:38.020332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0180017-e65e-4c1c-b741-2aa0be1c64d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:38.022090 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0180017-e65e-4c1c-b741-2aa0be1c64d0 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:38.033287 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1368/2748] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:35:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:35:47.849021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=74,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:35:48.029131 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=76,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:35:59.023171 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1a832-5dd2-4222-8218-e2ab7564dade None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:59.023689 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1a832-5dd2-4222-8218-e2ab7564dade None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:59.023786 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1a832-5dd2-4222-8218-e2ab7564dade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:59.170891 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-2fb1a832-5dd2-4222-8218-e2ab7564dade None None] MFA Rules not processed for user `63371d392c8c4b73939ee73a1627c8dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:35:59.172667 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-2fb1a832-5dd2-4222-8218-e2ab7564dade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:59.179763 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1381/2749] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:35:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:35:59.391239 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0180017-e65e-4c1c-b741-2aa0be1c64d0 service nova] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:59.458801 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0180017-e65e-4c1c-b741-2aa0be1c64d0 service nova] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:59.463402 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feca3137-c6f5-46c0-9a68-987d223f0d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:59.464085 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca3137-c6f5-46c0-9a68-987d223f0d16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:59.464370 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca3137-c6f5-46c0-9a68-987d223f0d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:59.464644 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca3137-c6f5-46c0-9a68-987d223f0d16 admin admin] PATH_INFO: `/v3/users/7b4826601da34342b2870b87e5ce8d9c` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:59.465788 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feca3137-c6f5-46c0-9a68-987d223f0d16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b4826601da34342b2870b87e5ce8d9c)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:59.467708 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feca3137-c6f5-46c0-9a68-987d223f0d16 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:59.515470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-feca3137-c6f5-46c0-9a68-987d223f0d16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b4826601da34342b2870b87e5ce8d9c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:59.523005 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-feca3137-c6f5-46c0-9a68-987d223f0d16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b4826601da34342b2870b87e5ce8d9c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:59.535155 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-feca3137-c6f5-46c0-9a68-987d223f0d16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b4826601da34342b2870b87e5ce8d9c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:59.542475 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-feca3137-c6f5-46c0-9a68-987d223f0d16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b4826601da34342b2870b87e5ce8d9c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:59.543108 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-feca3137-c6f5-46c0-9a68-987d223f0d16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b4826601da34342b2870b87e5ce8d9c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:35:59.552023 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1369/2750] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:35:59 2022] DELETE /identity/v3/users/7b4826601da34342b2870b87e5ce8d9c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:35:59.553756 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54ef8d-95de-42d0-99c0-351ce857378e None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:59.554097 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54ef8d-95de-42d0-99c0-351ce857378e None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:59.554377 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54ef8d-95de-42d0-99c0-351ce857378e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:59.573362 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-3a54ef8d-95de-42d0-99c0-351ce857378e None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:35:59.574564 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-3a54ef8d-95de-42d0-99c0-351ce857378e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:59.633078 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1382/2751] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:35:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:35:59.642874 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feca3137-c6f5-46c0-9a68-987d223f0d16 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:35:59.644368 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-feca3137-c6f5-46c0-9a68-987d223f0d16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:35:59.710268 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feca3137-c6f5-46c0-9a68-987d223f0d16 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:35:59.714661 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86d91d-c1d1-4bb9-aeea-bcbb625ddcb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:35:59.715372 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86d91d-c1d1-4bb9-aeea-bcbb625ddcb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:35:59.715658 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86d91d-c1d1-4bb9-aeea-bcbb625ddcb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:35:59.715931 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86d91d-c1d1-4bb9-aeea-bcbb625ddcb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:35:59.725145 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff86d91d-c1d1-4bb9-aeea-bcbb625ddcb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:35:59.726711 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff86d91d-c1d1-4bb9-aeea-bcbb625ddcb2 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:35:59.736769 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1370/2752] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:35:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 20:36:00.053102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a54ef8d-95de-42d0-99c0-351ce857378e None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:36:00.054917 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-3a54ef8d-95de-42d0-99c0-351ce857378e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:36:00.070549 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a54ef8d-95de-42d0-99c0-351ce857378e None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:36:00.076018 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e709fcee-63ba-4a4b-a0c7-df8d2121f8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:36:00.077407 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709fcee-63ba-4a4b-a0c7-df8d2121f8a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:00.077851 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709fcee-63ba-4a4b-a0c7-df8d2121f8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:00.078141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709fcee-63ba-4a4b-a0c7-df8d2121f8a2 admin admin] PATH_INFO: `/v3/projects/4d04d8b1ba794ad6be2ecee6fbdcadf3` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:00.079369 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e709fcee-63ba-4a4b-a0c7-df8d2121f8a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d04d8b1ba794ad6be2ecee6fbdcadf3)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:36:00.081217 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e709fcee-63ba-4a4b-a0c7-df8d2121f8a2 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:36:00.129969 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-e709fcee-63ba-4a4b-a0c7-df8d2121f8a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d04d8b1ba794ad6be2ecee6fbdcadf3', 'request_id': 'req-e709fcee-63ba-4a4b-a0c7-df8d2121f8a2'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:36:00.138935 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-e709fcee-63ba-4a4b-a0c7-df8d2121f8a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d04d8b1ba794ad6be2ecee6fbdcadf3', 'request_id': 'req-e709fcee-63ba-4a4b-a0c7-df8d2121f8a2'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:36:00.145000 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-e709fcee-63ba-4a4b-a0c7-df8d2121f8a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d04d8b1ba794ad6be2ecee6fbdcadf3', 'request_id': 'req-e709fcee-63ba-4a4b-a0c7-df8d2121f8a2'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:36:00.157328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-e709fcee-63ba-4a4b-a0c7-df8d2121f8a2 admin admin] The token cache is being invalidate because project 4d04d8b1ba794ad6be2ecee6fbdcadf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:36:00.157771 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-e709fcee-63ba-4a4b-a0c7-df8d2121f8a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:36:00.158455 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-e709fcee-63ba-4a4b-a0c7-df8d2121f8a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:36:00.160028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1383/2753] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:36:00 2022] DELETE /identity/v3/projects/4d04d8b1ba794ad6be2ecee6fbdcadf3 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:36:00.165066 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b509a-2379-4707-80c8-c26792c9746f None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:00.165394 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b509a-2379-4707-80c8-c26792c9746f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:00.165728 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b509a-2379-4707-80c8-c26792c9746f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:00.185230 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-2e8b509a-2379-4707-80c8-c26792c9746f None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:36:00.186387 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2e8b509a-2379-4707-80c8-c26792c9746f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:36:00.203838 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:36:00.251087 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1371/2754] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:36:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:36:00.258606 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acc6b2-a2bd-42d2-aaa0-8910e86367b5 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:00.258954 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acc6b2-a2bd-42d2-aaa0-8910e86367b5 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:00.259872 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acc6b2-a2bd-42d2-aaa0-8910e86367b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:00.280028 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-e4acc6b2-a2bd-42d2-aaa0-8910e86367b5 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:36:00.281205 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-e4acc6b2-a2bd-42d2-aaa0-8910e86367b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:36:00.289029 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1384/2755] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:36:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:36:00.293748 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b509a-2379-4707-80c8-c26792c9746f None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:36:00.304092 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b509a-2379-4707-80c8-c26792c9746f None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:36:00.309751 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989cbdd3-da58-4fb2-b8e3-2251a131f0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:36:00.310351 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cbdd3-da58-4fb2-b8e3-2251a131f0a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:00.310631 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cbdd3-da58-4fb2-b8e3-2251a131f0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:00.310903 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cbdd3-da58-4fb2-b8e3-2251a131f0a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:00.311547 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989cbdd3-da58-4fb2-b8e3-2251a131f0a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:36:00.311847 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989cbdd3-da58-4fb2-b8e3-2251a131f0a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:36:00.313485 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989cbdd3-da58-4fb2-b8e3-2251a131f0a8 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:36:00.328144 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1372/2756] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:36:00.333982 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4acc6b2-a2bd-42d2-aaa0-8910e86367b5 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:36:00.347863 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4acc6b2-a2bd-42d2-aaa0-8910e86367b5 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:36:00.352451 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25eaa1-f778-47ee-a779-19dfb1395593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:36:00.353102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25eaa1-f778-47ee-a779-19dfb1395593 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:00.353419 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25eaa1-f778-47ee-a779-19dfb1395593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:00.353689 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25eaa1-f778-47ee-a779-19dfb1395593 admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:00.354308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a25eaa1-f778-47ee-a779-19dfb1395593 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:36:00.356882 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a25eaa1-f778-47ee-a779-19dfb1395593 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:36:00.387144 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1385/2757] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:36:00 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:36:00.391453 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989cbdd3-da58-4fb2-b8e3-2251a131f0a8 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:36:00.402012 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989cbdd3-da58-4fb2-b8e3-2251a131f0a8 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:36:00.459625 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df067d9-18e3-427a-83c3-fbb2478ae34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:36:00.460238 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df067d9-18e3-427a-83c3-fbb2478ae34e admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:00.460539 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df067d9-18e3-427a-83c3-fbb2478ae34e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:00.460813 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df067d9-18e3-427a-83c3-fbb2478ae34e admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:00.461469 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df067d9-18e3-427a-83c3-fbb2478ae34e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:36:00.463412 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df067d9-18e3-427a-83c3-fbb2478ae34e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:36:00.484734 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1373/2758] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:36:00 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:36:00.488801 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25eaa1-f778-47ee-a779-19dfb1395593 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:36:00.499392 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25eaa1-f778-47ee-a779-19dfb1395593 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:36:00.503941 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd8236-cb63-430d-94cd-687c1164abf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:36:00.504535 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd8236-cb63-430d-94cd-687c1164abf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:00.504811 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd8236-cb63-430d-94cd-687c1164abf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:00.505083 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd8236-cb63-430d-94cd-687c1164abf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:00.505744 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fd8236-cb63-430d-94cd-687c1164abf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:36:00.507341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fd8236-cb63-430d-94cd-687c1164abf0 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:36:00.518564 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1386/2759] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:36:00.522583 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df067d9-18e3-427a-83c3-fbb2478ae34e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:36:00.533132 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df067d9-18e3-427a-83c3-fbb2478ae34e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:36:00.538223 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a76c1bb-6e0b-4cd2-94a0-ec7f23b0ce73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:36:00.538892 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76c1bb-6e0b-4cd2-94a0-ec7f23b0ce73 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:00.539175 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76c1bb-6e0b-4cd2-94a0-ec7f23b0ce73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:00.539473 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76c1bb-6e0b-4cd2-94a0-ec7f23b0ce73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:00.540108 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a76c1bb-6e0b-4cd2-94a0-ec7f23b0ce73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:36:00.541779 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a76c1bb-6e0b-4cd2-94a0-ec7f23b0ce73 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:36:00.553069 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1374/2760] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:36:00.558392 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd8236-cb63-430d-94cd-687c1164abf0 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:36:00.569173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd8236-cb63-430d-94cd-687c1164abf0 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:36:00.574520 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6894d858-2742-4a7e-9a76-ce19879154d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:36:00.575172 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894d858-2742-4a7e-9a76-ce19879154d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:00.575451 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894d858-2742-4a7e-9a76-ce19879154d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:00.575722 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894d858-2742-4a7e-9a76-ce19879154d5 admin admin] PATH_INFO: `/v3/projects/cf4867c5594a49e3978f762d93d2106a/users/b4084e58c07e4c0d81d3f3b14da37258/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:00.590582 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6894d858-2742-4a7e-9a76-ce19879154d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf4867c5594a49e3978f762d93d2106a, user_id=b4084e58c07e4c0d81d3f3b14da37258, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:36:00.593203 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6894d858-2742-4a7e-9a76-ce19879154d5 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:36:00.606806 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1387/2761] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:36:00 2022] PUT /identity/v3/projects/cf4867c5594a49e3978f762d93d2106a/users/b4084e58c07e4c0d81d3f3b14da37258/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:36:00.617474 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366df3d2-e606-470d-95dd-b234e4c2e0b9 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:00.617872 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366df3d2-e606-470d-95dd-b234e4c2e0b9 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:00.618149 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366df3d2-e606-470d-95dd-b234e4c2e0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:00.647492 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-366df3d2-e606-470d-95dd-b234e4c2e0b9 None None] MFA Rules not processed for user `b4084e58c07e4c0d81d3f3b14da37258`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:36:00.648762 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-366df3d2-e606-470d-95dd-b234e4c2e0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:36:00.714478 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1375/2762] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:36:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:36:00.724933 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6894d858-2742-4a7e-9a76-ce19879154d5 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:36:00.726647 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6894d858-2742-4a7e-9a76-ce19879154d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:36:00.742300 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:36:00.792118 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6894d858-2742-4a7e-9a76-ce19879154d5 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:36:00.796521 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad65d7e-56c1-4c17-9633-0a6c4cd6f913 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:36:00.797125 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad65d7e-56c1-4c17-9633-0a6c4cd6f913 service cinder] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:00.797462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad65d7e-56c1-4c17-9633-0a6c4cd6f913 service cinder] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:00.797742 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad65d7e-56c1-4c17-9633-0a6c4cd6f913 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:00.807486 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad65d7e-56c1-4c17-9633-0a6c4cd6f913 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:36:00.808909 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad65d7e-56c1-4c17-9633-0a6c4cd6f913 service cinder] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:36:00.817526 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1388/2763] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:36:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:36:10.710848 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:36:10.774107 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366df3d2-e606-470d-95dd-b234e4c2e0b9 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:36:10.813799 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=20 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:36:10.900994 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366df3d2-e606-470d-95dd-b234e4c2e0b9 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:36:10.905918 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db6f0d-e581-4fe9-9b5e-b475e4c303a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:36:10.906545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db6f0d-e581-4fe9-9b5e-b475e4c303a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:10.906823 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db6f0d-e581-4fe9-9b5e-b475e4c303a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:10.907126 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db6f0d-e581-4fe9-9b5e-b475e4c303a0 admin admin] PATH_INFO: `/v3/users/b4084e58c07e4c0d81d3f3b14da37258` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:10.908552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40db6f0d-e581-4fe9-9b5e-b475e4c303a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4084e58c07e4c0d81d3f3b14da37258)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:36:10.910580 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40db6f0d-e581-4fe9-9b5e-b475e4c303a0 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:36:10.958219 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-40db6f0d-e581-4fe9-9b5e-b475e4c303a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4084e58c07e4c0d81d3f3b14da37258'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:36:10.965895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-40db6f0d-e581-4fe9-9b5e-b475e4c303a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4084e58c07e4c0d81d3f3b14da37258'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:36:10.978627 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-40db6f0d-e581-4fe9-9b5e-b475e4c303a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4084e58c07e4c0d81d3f3b14da37258'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:36:10.986675 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-40db6f0d-e581-4fe9-9b5e-b475e4c303a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4084e58c07e4c0d81d3f3b14da37258'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:36:10.987306 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-40db6f0d-e581-4fe9-9b5e-b475e4c303a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4084e58c07e4c0d81d3f3b14da37258'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:36:10.993443 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1376/2764] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:36:10 2022] DELETE /identity/v3/users/b4084e58c07e4c0d81d3f3b14da37258 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:36:10.998184 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546c00b-105d-487b-98dd-82185a18d533 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:10.998470 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546c00b-105d-487b-98dd-82185a18d533 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:10.998747 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546c00b-105d-487b-98dd-82185a18d533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:11.048449 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-0546c00b-105d-487b-98dd-82185a18d533 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:36:11.049695 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-0546c00b-105d-487b-98dd-82185a18d533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:36:11.112317 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1389/2765] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:36:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:36:11.122095 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db6f0d-e581-4fe9-9b5e-b475e4c303a0 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:36:11.123548 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-40db6f0d-e581-4fe9-9b5e-b475e4c303a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:36:11.174153 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:36:11.214648 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db6f0d-e581-4fe9-9b5e-b475e4c303a0 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:36:11.219331 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ac923-7d22-4908-a724-ce71b5cdd06b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:36:11.220069 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ac923-7d22-4908-a724-ce71b5cdd06b service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:11.220677 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ac923-7d22-4908-a724-ce71b5cdd06b service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:11.221011 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ac923-7d22-4908-a724-ce71b5cdd06b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:11.233649 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ac923-7d22-4908-a724-ce71b5cdd06b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:36:11.235371 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ac923-7d22-4908-a724-ce71b5cdd06b service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:36:11.246274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1377/2766] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:36:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:36:11.638512 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546c00b-105d-487b-98dd-82185a18d533 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:36:11.640040 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-0546c00b-105d-487b-98dd-82185a18d533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:36:11.653733 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546c00b-105d-487b-98dd-82185a18d533 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:36:11.663885 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82274793-835c-4b5f-8ad0-d894de1d6fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:36:11.664401 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82274793-835c-4b5f-8ad0-d894de1d6fe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:11.664724 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82274793-835c-4b5f-8ad0-d894de1d6fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:11.665524 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82274793-835c-4b5f-8ad0-d894de1d6fe6 admin admin] PATH_INFO: `/v3/projects/cf4867c5594a49e3978f762d93d2106a` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:11.673352 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82274793-835c-4b5f-8ad0-d894de1d6fe6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf4867c5594a49e3978f762d93d2106a)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:36:11.673352 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82274793-835c-4b5f-8ad0-d894de1d6fe6 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:36:11.726136 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-82274793-835c-4b5f-8ad0-d894de1d6fe6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf4867c5594a49e3978f762d93d2106a', 'request_id': 'req-82274793-835c-4b5f-8ad0-d894de1d6fe6'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:36:11.735837 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-82274793-835c-4b5f-8ad0-d894de1d6fe6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf4867c5594a49e3978f762d93d2106a', 'request_id': 'req-82274793-835c-4b5f-8ad0-d894de1d6fe6'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:36:11.743844 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-82274793-835c-4b5f-8ad0-d894de1d6fe6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf4867c5594a49e3978f762d93d2106a', 'request_id': 'req-82274793-835c-4b5f-8ad0-d894de1d6fe6'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:36:11.756238 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-82274793-835c-4b5f-8ad0-d894de1d6fe6 admin admin] The token cache is being invalidate because project cf4867c5594a49e3978f762d93d2106a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:36:11.756572 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-82274793-835c-4b5f-8ad0-d894de1d6fe6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:36:11.757231 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-82274793-835c-4b5f-8ad0-d894de1d6fe6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:36:11.759383 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1390/2767] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:36:11 2022] DELETE /identity/v3/projects/cf4867c5594a49e3978f762d93d2106a => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:36:11.764311 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7c509-11fb-46e4-8595-6ae3b57a35d2 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:11.764862 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7c509-11fb-46e4-8595-6ae3b57a35d2 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:11.765254 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7c509-11fb-46e4-8595-6ae3b57a35d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:11.822424 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-23d7c509-11fb-46e4-8595-6ae3b57a35d2 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:36:11.823622 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-23d7c509-11fb-46e4-8595-6ae3b57a35d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:36:11.885304 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1378/2768] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:36:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:36:11.956900 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b47062-a659-4bc3-98d9-395f5f28b02d None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:11.957511 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b47062-a659-4bc3-98d9-395f5f28b02d None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:11.957511 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b47062-a659-4bc3-98d9-395f5f28b02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:11.976467 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-f5b47062-a659-4bc3-98d9-395f5f28b02d None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:36:11.978194 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-f5b47062-a659-4bc3-98d9-395f5f28b02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:36:11.986110 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1391/2769] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:36:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:36:11.990238 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7c509-11fb-46e4-8595-6ae3b57a35d2 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:36:12.000736 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7c509-11fb-46e4-8595-6ae3b57a35d2 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:36:12.005539 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36226290-fa89-4f6c-87ac-77e47ade6425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:36:12.006165 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36226290-fa89-4f6c-87ac-77e47ade6425 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:12.006446 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36226290-fa89-4f6c-87ac-77e47ade6425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:12.006719 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36226290-fa89-4f6c-87ac-77e47ade6425 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:12.007370 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36226290-fa89-4f6c-87ac-77e47ade6425 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:36:12.007699 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36226290-fa89-4f6c-87ac-77e47ade6425 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:36:12.009310 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36226290-fa89-4f6c-87ac-77e47ade6425 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:36:12.020730 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1379/2770] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:36:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:36:12.024972 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b47062-a659-4bc3-98d9-395f5f28b02d None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:36:12.033897 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b47062-a659-4bc3-98d9-395f5f28b02d None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:36:12.038435 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3571b0-ccb3-4534-9caa-d71e07693bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:36:12.039014 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3571b0-ccb3-4534-9caa-d71e07693bae admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:12.039287 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3571b0-ccb3-4534-9caa-d71e07693bae admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:12.039549 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3571b0-ccb3-4534-9caa-d71e07693bae admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:12.040659 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3571b0-ccb3-4534-9caa-d71e07693bae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:36:12.042533 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3571b0-ccb3-4534-9caa-d71e07693bae admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:36:12.069878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1392/2771] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:36:12 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:36:12.073899 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36226290-fa89-4f6c-87ac-77e47ade6425 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:36:12.083515 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36226290-fa89-4f6c-87ac-77e47ade6425 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:36:12.139150 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774b70ff-725d-4a3c-9420-d566dbc1b7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:36:12.139777 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b70ff-725d-4a3c-9420-d566dbc1b7db admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:12.140139 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b70ff-725d-4a3c-9420-d566dbc1b7db admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:12.140444 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b70ff-725d-4a3c-9420-d566dbc1b7db admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:12.141070 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774b70ff-725d-4a3c-9420-d566dbc1b7db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:36:12.143059 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774b70ff-725d-4a3c-9420-d566dbc1b7db admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:36:12.164593 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1380/2772] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:36:12 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:36:12.169457 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3571b0-ccb3-4534-9caa-d71e07693bae admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:36:12.179553 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3571b0-ccb3-4534-9caa-d71e07693bae admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:36:12.184706 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ad38a-e1b4-45d8-98a4-317400a4b2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:36:12.185321 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ad38a-e1b4-45d8-98a4-317400a4b2cd admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:12.185602 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ad38a-e1b4-45d8-98a4-317400a4b2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:12.185872 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ad38a-e1b4-45d8-98a4-317400a4b2cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:12.186504 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319ad38a-e1b4-45d8-98a4-317400a4b2cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:36:12.188070 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319ad38a-e1b4-45d8-98a4-317400a4b2cd admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:36:12.198475 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1393/2773] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:36:12.202400 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774b70ff-725d-4a3c-9420-d566dbc1b7db admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:36:12.213392 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774b70ff-725d-4a3c-9420-d566dbc1b7db admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:36:12.218025 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a0006-aac4-4a70-afad-a8aa52acbf32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:36:12.218620 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a0006-aac4-4a70-afad-a8aa52acbf32 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:12.218906 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a0006-aac4-4a70-afad-a8aa52acbf32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:12.219182 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a0006-aac4-4a70-afad-a8aa52acbf32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:12.219815 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a0006-aac4-4a70-afad-a8aa52acbf32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:36:12.221441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a0006-aac4-4a70-afad-a8aa52acbf32 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:36:12.232454 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1381/2774] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:36:12.237013 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ad38a-e1b4-45d8-98a4-317400a4b2cd admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:36:12.246810 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ad38a-e1b4-45d8-98a4-317400a4b2cd admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:36:12.251534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd843d2-ca36-4cbc-b386-3137e4e44889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:36:12.252237 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd843d2-ca36-4cbc-b386-3137e4e44889 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:12.252519 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd843d2-ca36-4cbc-b386-3137e4e44889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:12.252793 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd843d2-ca36-4cbc-b386-3137e4e44889 admin admin] PATH_INFO: `/v3/projects/474aa97f67a8457ab6900081711a0bf1/users/3540adbd8efb4a7a97ca44a6c0e5e51d/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:12.267215 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd843d2-ca36-4cbc-b386-3137e4e44889 admin admin] RBAC: Authorizing `identity:create_grant(project_id=474aa97f67a8457ab6900081711a0bf1, user_id=3540adbd8efb4a7a97ca44a6c0e5e51d, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:36:12.269884 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd843d2-ca36-4cbc-b386-3137e4e44889 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:36:12.272006 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:36:12.284619 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1394/2775] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:36:12 2022] PUT /identity/v3/projects/474aa97f67a8457ab6900081711a0bf1/users/3540adbd8efb4a7a97ca44a6c0e5e51d/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:36:12.289734 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ca04b-3f77-4b8f-8bd7-522b10e18627 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:12.290589 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ca04b-3f77-4b8f-8bd7-522b10e18627 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:12.290864 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ca04b-3f77-4b8f-8bd7-522b10e18627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:12.304296 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:36:12.311691 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-c50ca04b-3f77-4b8f-8bd7-522b10e18627 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:36:12.313525 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c50ca04b-3f77-4b8f-8bd7-522b10e18627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:36:12.372490 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1382/2776] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:36:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:36:12.382027 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd843d2-ca36-4cbc-b386-3137e4e44889 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:36:12.383757 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6cd843d2-ca36-4cbc-b386-3137e4e44889 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:36:12.439749 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd843d2-ca36-4cbc-b386-3137e4e44889 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:36:12.443262 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f02286-408b-400e-8bd8-3f45f1495e70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:36:12.443855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f02286-408b-400e-8bd8-3f45f1495e70 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:12.444128 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f02286-408b-400e-8bd8-3f45f1495e70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:12.444395 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f02286-408b-400e-8bd8-3f45f1495e70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:12.454030 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f02286-408b-400e-8bd8-3f45f1495e70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:36:12.455834 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f02286-408b-400e-8bd8-3f45f1495e70 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:36:12.467471 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1395/2777] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:36:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:36:13.681221 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ca04b-3f77-4b8f-8bd7-522b10e18627 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:36:13.682816 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c50ca04b-3f77-4b8f-8bd7-522b10e18627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:36:13.790858 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ca04b-3f77-4b8f-8bd7-522b10e18627 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:36:13.796614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d45d79-7c7b-4df6-9e65-d95375b43bf8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26d24cf0a21d491082b0c2f11e749446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:36:13.797382 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d45d79-7c7b-4df6-9e65-d95375b43bf8 service placement] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:13.797660 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d45d79-7c7b-4df6-9e65-d95375b43bf8 service placement] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:13.797930 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d45d79-7c7b-4df6-9e65-d95375b43bf8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:13.800079 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-38d45d79-7c7b-4df6-9e65-d95375b43bf8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:36:13.820547 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d45d79-7c7b-4df6-9e65-d95375b43bf8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:36:13.822262 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d45d79-7c7b-4df6-9e65-d95375b43bf8 service placement] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:36:13.833286 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1383/2778] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:36:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:36:15.792713 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f02286-408b-400e-8bd8-3f45f1495e70 service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:36:15.794581 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-69f02286-408b-400e-8bd8-3f45f1495e70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:36:15.807564 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f02286-408b-400e-8bd8-3f45f1495e70 service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:36:15.811776 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba01d9b8-ef1b-42a7-af42-46d63bfedde9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:36:15.812716 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01d9b8-ef1b-42a7-af42-46d63bfedde9 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:15.813091 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01d9b8-ef1b-42a7-af42-46d63bfedde9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:15.813491 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01d9b8-ef1b-42a7-af42-46d63bfedde9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:15.815430 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-ba01d9b8-ef1b-42a7-af42-46d63bfedde9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:36:15.900909 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba01d9b8-ef1b-42a7-af42-46d63bfedde9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:36:15.902797 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba01d9b8-ef1b-42a7-af42-46d63bfedde9 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:36:15.915515 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1396/2779] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:36:15 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:36:16.839463 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094801f2-6a99-4f1f-960e-c215857a4cbd None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:16.839968 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094801f2-6a99-4f1f-960e-c215857a4cbd None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:16.840173 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094801f2-6a99-4f1f-960e-c215857a4cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:16.884948 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-094801f2-6a99-4f1f-960e-c215857a4cbd None None] MFA Rules not processed for user `3540adbd8efb4a7a97ca44a6c0e5e51d`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:36:16.886590 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-094801f2-6a99-4f1f-960e-c215857a4cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:36:17.001285 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1384/2780] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:36:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:36:17.005637 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba01d9b8-ef1b-42a7-af42-46d63bfedde9 service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:36:17.017866 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba01d9b8-ef1b-42a7-af42-46d63bfedde9 service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:36:17.023307 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb196850-51dd-4157-93cd-7783e26261d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:36:17.023926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb196850-51dd-4157-93cd-7783e26261d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:17.024874 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb196850-51dd-4157-93cd-7783e26261d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:17.025159 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb196850-51dd-4157-93cd-7783e26261d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:17.025833 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb196850-51dd-4157-93cd-7783e26261d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:36:17.027774 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb196850-51dd-4157-93cd-7783e26261d5 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:36:17.061477 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1397/2781] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:36:17 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:36:17.065631 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094801f2-6a99-4f1f-960e-c215857a4cbd None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:36:17.077392 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094801f2-6a99-4f1f-960e-c215857a4cbd None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:36:17.081130 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:36:17.163049 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19845992-003c-48e6-9874-fd0a00af752e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:36:17.165859 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19845992-003c-48e6-9874-fd0a00af752e admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:17.166623 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19845992-003c-48e6-9874-fd0a00af752e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:17.167260 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19845992-003c-48e6-9874-fd0a00af752e admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:17.168156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19845992-003c-48e6-9874-fd0a00af752e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:36:17.170451 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19845992-003c-48e6-9874-fd0a00af752e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:36:17.206742 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1385/2782] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:36:17 2022] POST /identity/v3/users => generated 293 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:36:17.210746 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb196850-51dd-4157-93cd-7783e26261d5 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:36:17.223668 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb196850-51dd-4157-93cd-7783e26261d5 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:36:17.242433 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b027e92-e9cb-4dc2-bb11-aa8144bbffa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:36:17.243118 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b027e92-e9cb-4dc2-bb11-aa8144bbffa5 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:17.243395 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b027e92-e9cb-4dc2-bb11-aa8144bbffa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:17.243663 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b027e92-e9cb-4dc2-bb11-aa8144bbffa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:17.244977 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b027e92-e9cb-4dc2-bb11-aa8144bbffa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:36:17.246949 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b027e92-e9cb-4dc2-bb11-aa8144bbffa5 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:36:17.263577 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1398/2783] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:36:17.267435 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19845992-003c-48e6-9874-fd0a00af752e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:36:17.279932 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19845992-003c-48e6-9874-fd0a00af752e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:36:17.286319 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0df2c9-b452-4084-8ef0-94946bd206b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:36:17.287163 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0df2c9-b452-4084-8ef0-94946bd206b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:17.287448 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0df2c9-b452-4084-8ef0-94946bd206b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:17.287721 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0df2c9-b452-4084-8ef0-94946bd206b0 admin admin] PATH_INFO: `/v3/projects/cdecc6100c5b4f7385eb7b83331aafbe/users/df09a37fed3140d2a07663e2425dc463/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:17.313232 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0df2c9-b452-4084-8ef0-94946bd206b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdecc6100c5b4f7385eb7b83331aafbe, user_id=df09a37fed3140d2a07663e2425dc463, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:36:17.316011 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0df2c9-b452-4084-8ef0-94946bd206b0 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:36:17.347620 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1386/2784] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:36:17 2022] PUT /identity/v3/projects/cdecc6100c5b4f7385eb7b83331aafbe/users/df09a37fed3140d2a07663e2425dc463/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:36:17.349106 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b027e92-e9cb-4dc2-bb11-aa8144bbffa5 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:36:17.361835 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b027e92-e9cb-4dc2-bb11-aa8144bbffa5 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:36:17.432690 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:36:17.439013 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a1b92-cb56-4124-b4fd-72b260e48c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:36:17.440051 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a1b92-cb56-4124-b4fd-72b260e48c60 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:17.441811 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a1b92-cb56-4124-b4fd-72b260e48c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:17.441811 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a1b92-cb56-4124-b4fd-72b260e48c60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:17.442457 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a1b92-cb56-4124-b4fd-72b260e48c60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:36:17.451162 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a1b92-cb56-4124-b4fd-72b260e48c60 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:36:17.479166 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1399/2785] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:36:17.483389 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0df2c9-b452-4084-8ef0-94946bd206b0 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:36:17.494870 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0df2c9-b452-4084-8ef0-94946bd206b0 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:36:17.499529 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26016693-7f28-42fa-8741-d9a117a40360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:36:17.500186 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26016693-7f28-42fa-8741-d9a117a40360 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:17.500467 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26016693-7f28-42fa-8741-d9a117a40360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:17.500740 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26016693-7f28-42fa-8741-d9a117a40360 admin admin] PATH_INFO: `/v3/projects/cdecc6100c5b4f7385eb7b83331aafbe/users/df09a37fed3140d2a07663e2425dc463/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:17.502705 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26016693-7f28-42fa-8741-d9a117a40360 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdecc6100c5b4f7385eb7b83331aafbe, user_id=df09a37fed3140d2a07663e2425dc463, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:36:17.505353 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26016693-7f28-42fa-8741-d9a117a40360 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:36:17.522864 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1387/2786] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:36:17 2022] PUT /identity/v3/projects/cdecc6100c5b4f7385eb7b83331aafbe/users/df09a37fed3140d2a07663e2425dc463/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:36:22.063078 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca03f55-fb8a-4d2b-aad6-242727d98009 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:22.063647 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca03f55-fb8a-4d2b-aad6-242727d98009 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:22.063647 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca03f55-fb8a-4d2b-aad6-242727d98009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:22.092693 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-3ca03f55-fb8a-4d2b-aad6-242727d98009 None None] MFA Rules not processed for user `df09a37fed3140d2a07663e2425dc463`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:36:22.093928 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-3ca03f55-fb8a-4d2b-aad6-242727d98009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:36:22.163694 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1400/2787] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:36:22 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:36:22.179562 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26016693-7f28-42fa-8741-d9a117a40360 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:36:22.181467 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-26016693-7f28-42fa-8741-d9a117a40360 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:36:22.287249 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26016693-7f28-42fa-8741-d9a117a40360 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:36:22.293602 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd45e6d-e13b-425b-a61b-425b949ee571 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:36:22.294215 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd45e6d-e13b-425b-a61b-425b949ee571 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:22.294495 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd45e6d-e13b-425b-a61b-425b949ee571 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:22.294768 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd45e6d-e13b-425b-a61b-425b949ee571 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:22.324665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:36:22.353916 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd45e6d-e13b-425b-a61b-425b949ee571 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:36:22.359586 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd45e6d-e13b-425b-a61b-425b949ee571 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:36:22.370439 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1388/2788] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:36:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:36:23.440129 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef07343-005e-4739-b32b-05df4a7ab9c1 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:23.440612 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef07343-005e-4739-b32b-05df4a7ab9c1 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:23.440712 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef07343-005e-4739-b32b-05df4a7ab9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:23.459613 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-6ef07343-005e-4739-b32b-05df4a7ab9c1 None None] MFA Rules not processed for user `3540adbd8efb4a7a97ca44a6c0e5e51d`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:36:23.460788 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6ef07343-005e-4739-b32b-05df4a7ab9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:36:23.468051 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1401/2789] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:36:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:36:32.366083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=14 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:36:33.454289 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=40 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:36:41.448871 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a61914e-ae35-43b7-ad7b-7b713efe5134 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:41.449460 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a61914e-ae35-43b7-ad7b-7b713efe5134 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:41.449460 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a61914e-ae35-43b7-ad7b-7b713efe5134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:41.471819 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-4a61914e-ae35-43b7-ad7b-7b713efe5134 None None] MFA Rules not processed for user `f0eca860c013457ea79e4633f3e6d895`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:36:41.473074 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4a61914e-ae35-43b7-ad7b-7b713efe5134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:36:41.532154 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1389/2790] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:36:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:36:51.525307 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=25 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:36:56.501949 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef07343-005e-4739-b32b-05df4a7ab9c1 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:36:56.570139 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef07343-005e-4739-b32b-05df4a7ab9c1 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:36:56.573612 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488245f8-0197-4f55-8a87-597c86b0c03b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:36:56.574199 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488245f8-0197-4f55-8a87-597c86b0c03b service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:36:56.574470 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488245f8-0197-4f55-8a87-597c86b0c03b service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:36:56.574741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488245f8-0197-4f55-8a87-597c86b0c03b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:36:56.583331 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488245f8-0197-4f55-8a87-597c86b0c03b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:36:56.585161 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488245f8-0197-4f55-8a87-597c86b0c03b service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:36:56.594121 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1402/2791] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:36:56 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:37:06.590179 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=30 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:37:52.926558 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a61914e-ae35-43b7-ad7b-7b713efe5134 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:37:52.927997 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4a61914e-ae35-43b7-ad7b-7b713efe5134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:37:52.993856 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a61914e-ae35-43b7-ad7b-7b713efe5134 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:37:52.998430 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5eb117-6c3e-4bf8-a4e6-0e9b69be28a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:37:52.999068 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5eb117-6c3e-4bf8-a4e6-0e9b69be28a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:37:52.999358 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5eb117-6c3e-4bf8-a4e6-0e9b69be28a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:37:52.999635 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5eb117-6c3e-4bf8-a4e6-0e9b69be28a2 admin admin] PATH_INFO: `/v3/users/f0eca860c013457ea79e4633f3e6d895` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:37:53.000793 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5eb117-6c3e-4bf8-a4e6-0e9b69be28a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0eca860c013457ea79e4633f3e6d895)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:37:53.002736 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5eb117-6c3e-4bf8-a4e6-0e9b69be28a2 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:37:53.049536 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-cf5eb117-6c3e-4bf8-a4e6-0e9b69be28a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0eca860c013457ea79e4633f3e6d895'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:37:53.056547 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-cf5eb117-6c3e-4bf8-a4e6-0e9b69be28a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0eca860c013457ea79e4633f3e6d895'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:37:53.068840 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-cf5eb117-6c3e-4bf8-a4e6-0e9b69be28a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0eca860c013457ea79e4633f3e6d895'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:37:53.076114 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-cf5eb117-6c3e-4bf8-a4e6-0e9b69be28a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0eca860c013457ea79e4633f3e6d895'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:37:53.076814 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-cf5eb117-6c3e-4bf8-a4e6-0e9b69be28a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0eca860c013457ea79e4633f3e6d895'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:37:53.083233 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1390/2792] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:37:52 2022] DELETE /identity/v3/users/f0eca860c013457ea79e4633f3e6d895 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:37:53.253031 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488245f8-0197-4f55-8a87-597c86b0c03b service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:37:53.268880 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488245f8-0197-4f55-8a87-597c86b0c03b service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:37:53.326663 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc94ecf-bea4-4c75-b40c-5f3c45ce72a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:37:53.327282 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc94ecf-bea4-4c75-b40c-5f3c45ce72a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:37:53.327557 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc94ecf-bea4-4c75-b40c-5f3c45ce72a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:37:53.327840 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc94ecf-bea4-4c75-b40c-5f3c45ce72a9 admin admin] PATH_INFO: `/v3/projects/624ed22fc46047e1a842200db3388b63` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:37:53.328906 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc94ecf-bea4-4c75-b40c-5f3c45ce72a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=624ed22fc46047e1a842200db3388b63)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:37:53.330841 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc94ecf-bea4-4c75-b40c-5f3c45ce72a9 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:37:53.375398 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-8cc94ecf-bea4-4c75-b40c-5f3c45ce72a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '624ed22fc46047e1a842200db3388b63', 'request_id': 'req-8cc94ecf-bea4-4c75-b40c-5f3c45ce72a9'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:37:53.382803 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-8cc94ecf-bea4-4c75-b40c-5f3c45ce72a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '624ed22fc46047e1a842200db3388b63', 'request_id': 'req-8cc94ecf-bea4-4c75-b40c-5f3c45ce72a9'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:37:53.388246 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-8cc94ecf-bea4-4c75-b40c-5f3c45ce72a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '624ed22fc46047e1a842200db3388b63', 'request_id': 'req-8cc94ecf-bea4-4c75-b40c-5f3c45ce72a9'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:37:53.399929 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-8cc94ecf-bea4-4c75-b40c-5f3c45ce72a9 admin admin] The token cache is being invalidate because project 624ed22fc46047e1a842200db3388b63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:37:53.400248 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-8cc94ecf-bea4-4c75-b40c-5f3c45ce72a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:37:53.400882 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-8cc94ecf-bea4-4c75-b40c-5f3c45ce72a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:37:53.402359 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1403/2793] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:37:53 2022] DELETE /identity/v3/projects/624ed22fc46047e1a842200db3388b63 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:37:53.407243 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb308df-a4d3-4506-8000-9e9eccb7dbf9 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:37:53.407538 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb308df-a4d3-4506-8000-9e9eccb7dbf9 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:37:53.407817 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb308df-a4d3-4506-8000-9e9eccb7dbf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:37:53.427176 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-4cb308df-a4d3-4506-8000-9e9eccb7dbf9 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:37:53.428306 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4cb308df-a4d3-4506-8000-9e9eccb7dbf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:37:53.487804 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1391/2794] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:37:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:37:53.496125 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f378c5b1-6fe5-48e9-8f5d-e578d40d80d3 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:37:53.496411 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f378c5b1-6fe5-48e9-8f5d-e578d40d80d3 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:37:53.496710 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f378c5b1-6fe5-48e9-8f5d-e578d40d80d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:37:53.516401 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-f378c5b1-6fe5-48e9-8f5d-e578d40d80d3 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:37:53.517624 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-f378c5b1-6fe5-48e9-8f5d-e578d40d80d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:37:53.525685 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1404/2795] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:37:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:37:53.529687 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb308df-a4d3-4506-8000-9e9eccb7dbf9 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:37:53.539230 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb308df-a4d3-4506-8000-9e9eccb7dbf9 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:37:53.543707 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f86ff5-586a-41a0-84b9-93841405bfdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:37:53.544300 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f86ff5-586a-41a0-84b9-93841405bfdc admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:37:53.544577 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f86ff5-586a-41a0-84b9-93841405bfdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:37:53.544848 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f86ff5-586a-41a0-84b9-93841405bfdc admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:37:53.545515 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f86ff5-586a-41a0-84b9-93841405bfdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:37:53.545814 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f86ff5-586a-41a0-84b9-93841405bfdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:37:53.547381 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f86ff5-586a-41a0-84b9-93841405bfdc admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:37:53.559117 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1392/2796] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:37:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:37:53.563263 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f378c5b1-6fe5-48e9-8f5d-e578d40d80d3 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:37:53.573087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f378c5b1-6fe5-48e9-8f5d-e578d40d80d3 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:37:53.577784 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2f3a37-e3fa-4b2c-b053-7cf01e6c7acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:37:53.578456 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f3a37-e3fa-4b2c-b053-7cf01e6c7acb admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:37:53.578761 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f3a37-e3fa-4b2c-b053-7cf01e6c7acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:37:53.579035 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f3a37-e3fa-4b2c-b053-7cf01e6c7acb admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:37:53.579649 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2f3a37-e3fa-4b2c-b053-7cf01e6c7acb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:37:53.581579 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2f3a37-e3fa-4b2c-b053-7cf01e6c7acb admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:37:53.609836 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1405/2797] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:37:53 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:37:53.614453 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f86ff5-586a-41a0-84b9-93841405bfdc admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:37:53.624545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f86ff5-586a-41a0-84b9-93841405bfdc admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:37:53.628054 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:37:53.683060 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc301b5-9c16-493c-9353-d12c92bd3a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:37:53.683602 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc301b5-9c16-493c-9353-d12c92bd3a11 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:37:53.683882 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc301b5-9c16-493c-9353-d12c92bd3a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:37:53.684152 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc301b5-9c16-493c-9353-d12c92bd3a11 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:37:53.684769 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc301b5-9c16-493c-9353-d12c92bd3a11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:37:53.686695 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc301b5-9c16-493c-9353-d12c92bd3a11 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:37:53.707754 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1393/2798] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:37:53 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:37:53.711854 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2f3a37-e3fa-4b2c-b053-7cf01e6c7acb admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:37:53.721980 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2f3a37-e3fa-4b2c-b053-7cf01e6c7acb admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:37:53.726719 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365bb5de-3b35-4f16-a554-e1d70f3e8e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:37:53.727310 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365bb5de-3b35-4f16-a554-e1d70f3e8e77 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:37:53.727592 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365bb5de-3b35-4f16-a554-e1d70f3e8e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:37:53.727868 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365bb5de-3b35-4f16-a554-e1d70f3e8e77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:37:53.728503 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365bb5de-3b35-4f16-a554-e1d70f3e8e77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:37:53.730150 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365bb5de-3b35-4f16-a554-e1d70f3e8e77 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:37:53.741619 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1406/2799] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:37:53.745374 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc301b5-9c16-493c-9353-d12c92bd3a11 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:37:53.756403 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc301b5-9c16-493c-9353-d12c92bd3a11 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:37:53.761417 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd86b3-b8e9-4e1c-a81f-2157b4f65c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:37:53.762014 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd86b3-b8e9-4e1c-a81f-2157b4f65c6a admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:37:53.762295 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd86b3-b8e9-4e1c-a81f-2157b4f65c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:37:53.762570 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd86b3-b8e9-4e1c-a81f-2157b4f65c6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:37:53.763211 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd86b3-b8e9-4e1c-a81f-2157b4f65c6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:37:53.764799 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd86b3-b8e9-4e1c-a81f-2157b4f65c6a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:37:53.775292 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1394/2800] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:37:53.779405 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365bb5de-3b35-4f16-a554-e1d70f3e8e77 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:37:53.790689 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365bb5de-3b35-4f16-a554-e1d70f3e8e77 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:37:53.796032 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea11c3e-bd48-48d8-8428-81d3611f201a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:37:53.796696 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea11c3e-bd48-48d8-8428-81d3611f201a admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:37:53.796961 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea11c3e-bd48-48d8-8428-81d3611f201a admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:37:53.797230 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea11c3e-bd48-48d8-8428-81d3611f201a admin admin] PATH_INFO: `/v3/projects/4ff5e199265744139faaa93df17c4292/users/c754968435064e548f54fbf02616708a/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:37:53.812599 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea11c3e-bd48-48d8-8428-81d3611f201a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ff5e199265744139faaa93df17c4292, user_id=c754968435064e548f54fbf02616708a, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:37:53.815256 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea11c3e-bd48-48d8-8428-81d3611f201a admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:37:53.828270 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1407/2801] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:37:53 2022] PUT /identity/v3/projects/4ff5e199265744139faaa93df17c4292/users/c754968435064e548f54fbf02616708a/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:37:53.839461 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5422001-7337-4f19-b0b3-74aca6efce19 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:37:53.839747 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5422001-7337-4f19-b0b3-74aca6efce19 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:37:53.840022 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5422001-7337-4f19-b0b3-74aca6efce19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:37:53.868693 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-d5422001-7337-4f19-b0b3-74aca6efce19 None None] MFA Rules not processed for user `c754968435064e548f54fbf02616708a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:37:53.869944 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d5422001-7337-4f19-b0b3-74aca6efce19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:37:53.933331 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1395/2802] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:37:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:37:53.944869 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea11c3e-bd48-48d8-8428-81d3611f201a admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:37:53.946600 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-5ea11c3e-bd48-48d8-8428-81d3611f201a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:37:53.976980 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:37:54.019487 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea11c3e-bd48-48d8-8428-81d3611f201a admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:37:54.025061 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b80330-34f7-4eff-b377-beeeb3c6518b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:37:54.025710 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b80330-34f7-4eff-b377-beeeb3c6518b service nova] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:37:54.025990 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b80330-34f7-4eff-b377-beeeb3c6518b service nova] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:37:54.026259 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b80330-34f7-4eff-b377-beeeb3c6518b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:37:54.034002 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b80330-34f7-4eff-b377-beeeb3c6518b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:37:54.036084 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b80330-34f7-4eff-b377-beeeb3c6518b service nova] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:37:54.044538 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1408/2803] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:37:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:37:56.710962 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5422001-7337-4f19-b0b3-74aca6efce19 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:37:56.772664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5422001-7337-4f19-b0b3-74aca6efce19 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:37:56.777101 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e89b7-6246-4b17-b8de-3fb189906c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:37:56.777763 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e89b7-6246-4b17-b8de-3fb189906c18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:37:56.778045 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e89b7-6246-4b17-b8de-3fb189906c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:37:56.778315 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e89b7-6246-4b17-b8de-3fb189906c18 admin admin] PATH_INFO: `/v3/users/c754968435064e548f54fbf02616708a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:37:56.779425 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952e89b7-6246-4b17-b8de-3fb189906c18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c754968435064e548f54fbf02616708a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:37:56.781348 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952e89b7-6246-4b17-b8de-3fb189906c18 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:37:56.796450 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:37:56.832131 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-952e89b7-6246-4b17-b8de-3fb189906c18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c754968435064e548f54fbf02616708a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:37:56.839086 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-952e89b7-6246-4b17-b8de-3fb189906c18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c754968435064e548f54fbf02616708a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:37:56.852398 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-952e89b7-6246-4b17-b8de-3fb189906c18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c754968435064e548f54fbf02616708a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:37:56.859980 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-952e89b7-6246-4b17-b8de-3fb189906c18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c754968435064e548f54fbf02616708a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:37:56.860732 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-952e89b7-6246-4b17-b8de-3fb189906c18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c754968435064e548f54fbf02616708a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:37:56.867243 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1396/2804] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:37:56 2022] DELETE /identity/v3/users/c754968435064e548f54fbf02616708a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:37:56.871923 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054b550-8606-45ff-83ce-fa869aaaa1c9 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:37:56.872403 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054b550-8606-45ff-83ce-fa869aaaa1c9 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:37:56.872581 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054b550-8606-45ff-83ce-fa869aaaa1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:37:56.891004 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-5054b550-8606-45ff-83ce-fa869aaaa1c9 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:37:56.892190 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-5054b550-8606-45ff-83ce-fa869aaaa1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:37:56.949783 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1409/2805] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:37:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:37:56.959428 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e89b7-6246-4b17-b8de-3fb189906c18 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:37:56.960819 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-952e89b7-6246-4b17-b8de-3fb189906c18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:37:57.031146 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e89b7-6246-4b17-b8de-3fb189906c18 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:37:57.034791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a3ebe3-076f-4c96-b32d-28d41f5dcd0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:37:57.035914 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3ebe3-076f-4c96-b32d-28d41f5dcd0c service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:37:57.036193 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3ebe3-076f-4c96-b32d-28d41f5dcd0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:37:57.036467 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3ebe3-076f-4c96-b32d-28d41f5dcd0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:37:57.045356 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a3ebe3-076f-4c96-b32d-28d41f5dcd0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:37:57.046790 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a3ebe3-076f-4c96-b32d-28d41f5dcd0c service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:37:57.056620 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1397/2806] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:37:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:37:57.315936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054b550-8606-45ff-83ce-fa869aaaa1c9 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:37:57.317378 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-5054b550-8606-45ff-83ce-fa869aaaa1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:37:57.329517 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054b550-8606-45ff-83ce-fa869aaaa1c9 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:37:57.334673 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830e3992-e81c-4d95-b427-48e8076f9baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:37:57.335309 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e3992-e81c-4d95-b427-48e8076f9baa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:37:57.335588 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e3992-e81c-4d95-b427-48e8076f9baa admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:37:57.335857 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e3992-e81c-4d95-b427-48e8076f9baa admin admin] PATH_INFO: `/v3/projects/4ff5e199265744139faaa93df17c4292` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:37:57.336981 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830e3992-e81c-4d95-b427-48e8076f9baa admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ff5e199265744139faaa93df17c4292)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:37:57.338898 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830e3992-e81c-4d95-b427-48e8076f9baa admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:37:57.385294 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-830e3992-e81c-4d95-b427-48e8076f9baa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ff5e199265744139faaa93df17c4292', 'request_id': 'req-830e3992-e81c-4d95-b427-48e8076f9baa'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:37:57.392411 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-830e3992-e81c-4d95-b427-48e8076f9baa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ff5e199265744139faaa93df17c4292', 'request_id': 'req-830e3992-e81c-4d95-b427-48e8076f9baa'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:37:57.398790 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-830e3992-e81c-4d95-b427-48e8076f9baa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ff5e199265744139faaa93df17c4292', 'request_id': 'req-830e3992-e81c-4d95-b427-48e8076f9baa'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:37:57.409763 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-830e3992-e81c-4d95-b427-48e8076f9baa admin admin] The token cache is being invalidate because project 4ff5e199265744139faaa93df17c4292 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:37:57.410054 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-830e3992-e81c-4d95-b427-48e8076f9baa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:37:57.411301 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-830e3992-e81c-4d95-b427-48e8076f9baa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:37:57.412761 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1410/2807] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:37:57 2022] DELETE /identity/v3/projects/4ff5e199265744139faaa93df17c4292 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:37:57.417639 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89fe24-d662-4963-a099-a31f8dabcf8d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:37:57.417928 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89fe24-d662-4963-a099-a31f8dabcf8d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:37:57.418207 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89fe24-d662-4963-a099-a31f8dabcf8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:37:57.437372 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-de89fe24-d662-4963-a099-a31f8dabcf8d None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:37:57.438574 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-de89fe24-d662-4963-a099-a31f8dabcf8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:37:57.496564 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1398/2808] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:37:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:37:57.504660 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2690a-a7c7-4076-82de-c4e8948ae0f3 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:37:57.504953 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2690a-a7c7-4076-82de-c4e8948ae0f3 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:37:57.505267 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2690a-a7c7-4076-82de-c4e8948ae0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:37:57.524400 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-a4c2690a-a7c7-4076-82de-c4e8948ae0f3 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:37:57.525628 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a4c2690a-a7c7-4076-82de-c4e8948ae0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:37:57.533432 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1411/2809] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:37:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:37:57.537502 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89fe24-d662-4963-a099-a31f8dabcf8d None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:37:57.546884 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89fe24-d662-4963-a099-a31f8dabcf8d None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:37:57.551922 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a5ff9-1372-4c73-b0fe-30fed34815f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:37:57.552507 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a5ff9-1372-4c73-b0fe-30fed34815f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:37:57.552783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a5ff9-1372-4c73-b0fe-30fed34815f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:37:57.553050 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a5ff9-1372-4c73-b0fe-30fed34815f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:37:57.553841 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a5ff9-1372-4c73-b0fe-30fed34815f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:37:57.554137 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a5ff9-1372-4c73-b0fe-30fed34815f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:37:57.555692 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a5ff9-1372-4c73-b0fe-30fed34815f5 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:37:57.566943 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1399/2810] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:37:57.571168 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c2690a-a7c7-4076-82de-c4e8948ae0f3 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:37:57.580106 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c2690a-a7c7-4076-82de-c4e8948ae0f3 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:37:57.584690 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3dc182-a46a-4a63-a444-989d67a35885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:37:57.585305 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3dc182-a46a-4a63-a444-989d67a35885 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:37:57.585586 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3dc182-a46a-4a63-a444-989d67a35885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:37:57.585855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3dc182-a46a-4a63-a444-989d67a35885 admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:37:57.586465 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3dc182-a46a-4a63-a444-989d67a35885 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:37:57.588337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3dc182-a46a-4a63-a444-989d67a35885 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:37:57.618104 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:37:57.626456 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1412/2811] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:37:57 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:37:57.630470 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a5ff9-1372-4c73-b0fe-30fed34815f5 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:37:57.640409 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a5ff9-1372-4c73-b0fe-30fed34815f5 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:37:57.649279 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:37:57.699116 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4094764-90cd-47b8-b180-693a33fc09c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:37:57.699724 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4094764-90cd-47b8-b180-693a33fc09c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:37:57.699999 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4094764-90cd-47b8-b180-693a33fc09c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:37:57.700275 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4094764-90cd-47b8-b180-693a33fc09c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:37:57.700900 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4094764-90cd-47b8-b180-693a33fc09c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:37:57.702860 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4094764-90cd-47b8-b180-693a33fc09c5 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:37:57.723561 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1400/2812] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:37:57 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:37:57.727917 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3dc182-a46a-4a63-a444-989d67a35885 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:37:57.738398 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3dc182-a46a-4a63-a444-989d67a35885 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:37:57.743496 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5126f4bd-f1f9-4dc4-885d-98527d911668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:37:57.744089 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126f4bd-f1f9-4dc4-885d-98527d911668 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:37:57.744375 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126f4bd-f1f9-4dc4-885d-98527d911668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:37:57.744668 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126f4bd-f1f9-4dc4-885d-98527d911668 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:37:57.745428 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5126f4bd-f1f9-4dc4-885d-98527d911668 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:37:57.747063 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5126f4bd-f1f9-4dc4-885d-98527d911668 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:37:57.757017 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1413/2813] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:37:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:37:57.760931 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4094764-90cd-47b8-b180-693a33fc09c5 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:37:57.770776 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4094764-90cd-47b8-b180-693a33fc09c5 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:37:57.775380 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e9fef-d1ee-4759-81be-c6625bc540c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:37:57.775975 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e9fef-d1ee-4759-81be-c6625bc540c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:37:57.776265 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e9fef-d1ee-4759-81be-c6625bc540c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:37:57.776540 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e9fef-d1ee-4759-81be-c6625bc540c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:37:57.777176 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516e9fef-d1ee-4759-81be-c6625bc540c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:37:57.778823 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516e9fef-d1ee-4759-81be-c6625bc540c5 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:37:57.789804 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1401/2814] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:37:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:37:57.793651 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5126f4bd-f1f9-4dc4-885d-98527d911668 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:37:57.803431 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5126f4bd-f1f9-4dc4-885d-98527d911668 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:37:57.807957 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e98dfe-8c1d-4fea-a086-ed0dff0a2ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:37:57.808594 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e98dfe-8c1d-4fea-a086-ed0dff0a2ef3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:37:57.808874 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e98dfe-8c1d-4fea-a086-ed0dff0a2ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:37:57.809146 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e98dfe-8c1d-4fea-a086-ed0dff0a2ef3 admin admin] PATH_INFO: `/v3/projects/30255cdb6ac84e978bfef02b3af1c3e5/users/173e57ac21424ddf9ba0f8691714479a/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:37:57.827827 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e98dfe-8c1d-4fea-a086-ed0dff0a2ef3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30255cdb6ac84e978bfef02b3af1c3e5, user_id=173e57ac21424ddf9ba0f8691714479a, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:37:57.830507 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e98dfe-8c1d-4fea-a086-ed0dff0a2ef3 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:37:57.844457 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1414/2815] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:37:57 2022] PUT /identity/v3/projects/30255cdb6ac84e978bfef02b3af1c3e5/users/173e57ac21424ddf9ba0f8691714479a/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:37:57.854816 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7d382-26e8-4edf-8d4a-a6e6598c4b11 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:37:57.855107 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7d382-26e8-4edf-8d4a-a6e6598c4b11 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:37:57.855382 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7d382-26e8-4edf-8d4a-a6e6598c4b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:37:57.884379 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e9f7d382-26e8-4edf-8d4a-a6e6598c4b11 None None] MFA Rules not processed for user `173e57ac21424ddf9ba0f8691714479a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:37:57.885634 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e9f7d382-26e8-4edf-8d4a-a6e6598c4b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:37:57.947983 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1402/2816] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:37:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:37:57.952387 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e98dfe-8c1d-4fea-a086-ed0dff0a2ef3 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:37:58.013480 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e98dfe-8c1d-4fea-a086-ed0dff0a2ef3 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:37:58.017957 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bbee73-abd4-494f-b62d-f4afaa066769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:37:58.018550 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbee73-abd4-494f-b62d-f4afaa066769 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:37:58.018821 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbee73-abd4-494f-b62d-f4afaa066769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:37:58.019088 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbee73-abd4-494f-b62d-f4afaa066769 admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:37:58.019700 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bbee73-abd4-494f-b62d-f4afaa066769 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:37:58.021601 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bbee73-abd4-494f-b62d-f4afaa066769 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:37:58.047760 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1415/2817] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:37:57 2022] POST /identity/v3/projects => generated 373 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:37:58.052120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7d382-26e8-4edf-8d4a-a6e6598c4b11 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:37:58.062735 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7d382-26e8-4edf-8d4a-a6e6598c4b11 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:37:58.117686 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332a2c20-e933-4dd1-8973-241cef913e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:37:58.118436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a2c20-e933-4dd1-8973-241cef913e53 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:37:58.118718 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a2c20-e933-4dd1-8973-241cef913e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:37:58.118989 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a2c20-e933-4dd1-8973-241cef913e53 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:37:58.119612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332a2c20-e933-4dd1-8973-241cef913e53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:37:58.121521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332a2c20-e933-4dd1-8973-241cef913e53 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:37:58.142515 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1403/2818] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:37:58 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:37:58.146634 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bbee73-abd4-494f-b62d-f4afaa066769 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:37:58.156782 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bbee73-abd4-494f-b62d-f4afaa066769 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:37:58.161361 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d0cb8-dac4-4ca7-a8d6-c6dd091cf2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:37:58.161954 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d0cb8-dac4-4ca7-a8d6-c6dd091cf2f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:37:58.162233 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d0cb8-dac4-4ca7-a8d6-c6dd091cf2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:37:58.162504 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d0cb8-dac4-4ca7-a8d6-c6dd091cf2f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:37:58.163137 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0d0cb8-dac4-4ca7-a8d6-c6dd091cf2f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:37:58.164717 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0d0cb8-dac4-4ca7-a8d6-c6dd091cf2f4 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:37:58.175315 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1416/2819] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:37:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:37:58.179783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332a2c20-e933-4dd1-8973-241cef913e53 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:37:58.189521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332a2c20-e933-4dd1-8973-241cef913e53 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:37:58.194928 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f16037-d728-4205-a1b5-1eb4484b6672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:37:58.195585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f16037-d728-4205-a1b5-1eb4484b6672 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:37:58.195898 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f16037-d728-4205-a1b5-1eb4484b6672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:37:58.196153 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f16037-d728-4205-a1b5-1eb4484b6672 admin admin] PATH_INFO: `/v3/projects/695915870eb34441bc11328af6af15a5/users/77661616f6ff458b8c5a04b07e982d64/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:37:58.204688 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:37:58.211740 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f16037-d728-4205-a1b5-1eb4484b6672 admin admin] RBAC: Authorizing `identity:create_grant(project_id=695915870eb34441bc11328af6af15a5, user_id=77661616f6ff458b8c5a04b07e982d64, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:37:58.214555 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f16037-d728-4205-a1b5-1eb4484b6672 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:37:58.226281 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1404/2820] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:37:58 2022] PUT /identity/v3/projects/695915870eb34441bc11328af6af15a5/users/77661616f6ff458b8c5a04b07e982d64/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:37:58.230163 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d0cb8-dac4-4ca7-a8d6-c6dd091cf2f4 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:37:58.240830 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d0cb8-dac4-4ca7-a8d6-c6dd091cf2f4 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:37:58.295214 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f206fd-533e-48f7-a15e-43554f6089f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:37:58.295738 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f206fd-533e-48f7-a15e-43554f6089f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:37:58.296017 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f206fd-533e-48f7-a15e-43554f6089f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:37:58.296287 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f206fd-533e-48f7-a15e-43554f6089f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:37:58.296914 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f206fd-533e-48f7-a15e-43554f6089f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:37:58.298523 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f206fd-533e-48f7-a15e-43554f6089f7 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:37:58.307814 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1417/2821] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:37:58 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:37:58.312008 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f16037-d728-4205-a1b5-1eb4484b6672 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:37:58.322490 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f16037-d728-4205-a1b5-1eb4484b6672 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:37:58.327013 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3df58c-e5b2-4657-b09a-3e4efb558073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:37:58.327655 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3df58c-e5b2-4657-b09a-3e4efb558073 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:37:58.327936 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3df58c-e5b2-4657-b09a-3e4efb558073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:37:58.328211 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3df58c-e5b2-4657-b09a-3e4efb558073 admin admin] PATH_INFO: `/v3/projects/695915870eb34441bc11328af6af15a5/users/77661616f6ff458b8c5a04b07e982d64/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:37:58.330139 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3df58c-e5b2-4657-b09a-3e4efb558073 admin admin] RBAC: Authorizing `identity:create_grant(project_id=695915870eb34441bc11328af6af15a5, user_id=77661616f6ff458b8c5a04b07e982d64, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:37:58.333364 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3df58c-e5b2-4657-b09a-3e4efb558073 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:37:58.343241 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1405/2822] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:37:58 2022] PUT /identity/v3/projects/695915870eb34441bc11328af6af15a5/users/77661616f6ff458b8c5a04b07e982d64/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:37:58.354523 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52448553-7b7a-4682-b144-4dae39e490f6 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:37:58.354805 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52448553-7b7a-4682-b144-4dae39e490f6 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:37:58.355074 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52448553-7b7a-4682-b144-4dae39e490f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:37:58.382414 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-52448553-7b7a-4682-b144-4dae39e490f6 None None] MFA Rules not processed for user `77661616f6ff458b8c5a04b07e982d64`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:37:58.383599 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-52448553-7b7a-4682-b144-4dae39e490f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:37:58.402474 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:37:58.450374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1418/2823] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:37:58 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:37:58.461538 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3df58c-e5b2-4657-b09a-3e4efb558073 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:37:58.463224 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-bb3df58c-e5b2-4657-b09a-3e4efb558073 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:37:58.540971 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3df58c-e5b2-4657-b09a-3e4efb558073 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:37:58.544626 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff7fea3-381e-4ec1-8ac3-70998247e862 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:37:58.545290 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff7fea3-381e-4ec1-8ac3-70998247e862 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:37:58.545538 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff7fea3-381e-4ec1-8ac3-70998247e862 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:37:58.545812 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff7fea3-381e-4ec1-8ac3-70998247e862 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:37:58.554700 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff7fea3-381e-4ec1-8ac3-70998247e862 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:37:58.556139 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff7fea3-381e-4ec1-8ac3-70998247e862 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:37:58.565436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1406/2824] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:37:58 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:37:58.639458 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52448553-7b7a-4682-b144-4dae39e490f6 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:37:58.649041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52448553-7b7a-4682-b144-4dae39e490f6 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:37:58.652585 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b3de81-8ad4-45bc-b769-142b038761b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:37:58.653171 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3de81-8ad4-45bc-b769-142b038761b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:37:58.653489 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3de81-8ad4-45bc-b769-142b038761b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:37:58.653758 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3de81-8ad4-45bc-b769-142b038761b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:37:58.710462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b3de81-8ad4-45bc-b769-142b038761b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:37:58.711919 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b3de81-8ad4-45bc-b769-142b038761b4 service cinder] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:37:58.720630 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1419/2825] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:37:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:38:08.561248 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=45,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:38:08.716637 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=47 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:38:16.570671 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff7fea3-381e-4ec1-8ac3-70998247e862 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:16.574194 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-aff7fea3-381e-4ec1-8ac3-70998247e862 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:16.634394 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff7fea3-381e-4ec1-8ac3-70998247e862 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:16.638043 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90d032-e830-4061-9e0f-ea08b362418d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:16.638650 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90d032-e830-4061-9e0f-ea08b362418d service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:16.638953 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90d032-e830-4061-9e0f-ea08b362418d service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:16.639230 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90d032-e830-4061-9e0f-ea08b362418d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:16.640961 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-cb90d032-e830-4061-9e0f-ea08b362418d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:16.712827 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb90d032-e830-4061-9e0f-ea08b362418d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:16.714291 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb90d032-e830-4061-9e0f-ea08b362418d service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:16.725239 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1407/2826] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:38:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:38:17.912163 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b3de81-8ad4-45bc-b769-142b038761b4 service cinder] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:17.921420 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b3de81-8ad4-45bc-b769-142b038761b4 service cinder] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:17.924912 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c46468-3f83-4d9a-a3cc-0fd8e666d05a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:17.925542 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c46468-3f83-4d9a-a3cc-0fd8e666d05a service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:17.925822 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c46468-3f83-4d9a-a3cc-0fd8e666d05a service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:17.926091 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c46468-3f83-4d9a-a3cc-0fd8e666d05a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:17.928030 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-92c46468-3f83-4d9a-a3cc-0fd8e666d05a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:17.941185 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c46468-3f83-4d9a-a3cc-0fd8e666d05a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:17.943338 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c46468-3f83-4d9a-a3cc-0fd8e666d05a service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:17.956435 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1420/2827] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:38:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:38:25.797192 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041777ea-9b1b-4d3b-a121-49795c46bb4c None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:25.797711 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041777ea-9b1b-4d3b-a121-49795c46bb4c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:25.797821 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041777ea-9b1b-4d3b-a121-49795c46bb4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:25.817228 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-041777ea-9b1b-4d3b-a121-49795c46bb4c None None] MFA Rules not processed for user `63371d392c8c4b73939ee73a1627c8dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:38:25.818515 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-041777ea-9b1b-4d3b-a121-49795c46bb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:25.874780 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1408/2828] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:38:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:38:26.635557 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c46468-3f83-4d9a-a3cc-0fd8e666d05a service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:26.636988 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-92c46468-3f83-4d9a-a3cc-0fd8e666d05a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:26.729434 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c46468-3f83-4d9a-a3cc-0fd8e666d05a service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:26.734874 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d1e199-afc5-4a15-8c69-6dc52491efa0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26d24cf0a21d491082b0c2f11e749446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:26.735899 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1e199-afc5-4a15-8c69-6dc52491efa0 service placement] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:26.736173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1e199-afc5-4a15-8c69-6dc52491efa0 service placement] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:26.736452 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1e199-afc5-4a15-8c69-6dc52491efa0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:26.738387 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-b5d1e199-afc5-4a15-8c69-6dc52491efa0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:26.754130 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d1e199-afc5-4a15-8c69-6dc52491efa0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:26.755707 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d1e199-afc5-4a15-8c69-6dc52491efa0 service placement] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:26.766467 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1421/2829] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:38:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:38:29.349744 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041777ea-9b1b-4d3b-a121-49795c46bb4c None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:29.351156 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-041777ea-9b1b-4d3b-a121-49795c46bb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:29.363262 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041777ea-9b1b-4d3b-a121-49795c46bb4c None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:29.366779 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27aaa87-9e86-4513-9233-59238c89d19a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:29.367382 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27aaa87-9e86-4513-9233-59238c89d19a service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:29.367679 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27aaa87-9e86-4513-9233-59238c89d19a service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:29.367934 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27aaa87-9e86-4513-9233-59238c89d19a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:29.369621 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a27aaa87-9e86-4513-9233-59238c89d19a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:29.379390 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27aaa87-9e86-4513-9233-59238c89d19a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:29.380805 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27aaa87-9e86-4513-9233-59238c89d19a service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:29.388767 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1409/2830] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:38:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:38:36.762027 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=38 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:38:37.156792 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d1e199-afc5-4a15-8c69-6dc52491efa0 service placement] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:37.221328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d1e199-afc5-4a15-8c69-6dc52491efa0 service placement] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:37.226794 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94eb2a-1fa4-4dba-b7d2-58ba7d06e5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:37.227510 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94eb2a-1fa4-4dba-b7d2-58ba7d06e5f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:37.227861 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94eb2a-1fa4-4dba-b7d2-58ba7d06e5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:37.228215 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94eb2a-1fa4-4dba-b7d2-58ba7d06e5f7 admin admin] PATH_INFO: `/v3/users/173e57ac21424ddf9ba0f8691714479a` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:37.229450 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e94eb2a-1fa4-4dba-b7d2-58ba7d06e5f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=173e57ac21424ddf9ba0f8691714479a)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:37.231519 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e94eb2a-1fa4-4dba-b7d2-58ba7d06e5f7 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:37.276599 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-7e94eb2a-1fa4-4dba-b7d2-58ba7d06e5f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '173e57ac21424ddf9ba0f8691714479a'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:37.289158 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-7e94eb2a-1fa4-4dba-b7d2-58ba7d06e5f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '173e57ac21424ddf9ba0f8691714479a'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:37.296197 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-7e94eb2a-1fa4-4dba-b7d2-58ba7d06e5f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '173e57ac21424ddf9ba0f8691714479a'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:37.304126 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-7e94eb2a-1fa4-4dba-b7d2-58ba7d06e5f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '173e57ac21424ddf9ba0f8691714479a'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:37.304792 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-7e94eb2a-1fa4-4dba-b7d2-58ba7d06e5f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '173e57ac21424ddf9ba0f8691714479a'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:37.311508 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1422/2831] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:38:37 2022] DELETE /identity/v3/users/173e57ac21424ddf9ba0f8691714479a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:38:37.315992 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82b96c-fe14-47e8-8e8d-199597494e56 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:37.317019 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82b96c-fe14-47e8-8e8d-199597494e56 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:37.317117 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82b96c-fe14-47e8-8e8d-199597494e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:37.337460 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-9a82b96c-fe14-47e8-8e8d-199597494e56 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:38:37.338674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9a82b96c-fe14-47e8-8e8d-199597494e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:37.368238 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:38:37.401900 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1410/2832] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:38:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:38:37.413120 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94eb2a-1fa4-4dba-b7d2-58ba7d06e5f7 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:37.414852 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7e94eb2a-1fa4-4dba-b7d2-58ba7d06e5f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:37.473395 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94eb2a-1fa4-4dba-b7d2-58ba7d06e5f7 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:37.477070 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f126351-88d2-4f0b-8c3a-b433386bb484 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:37.477707 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f126351-88d2-4f0b-8c3a-b433386bb484 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:37.477991 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f126351-88d2-4f0b-8c3a-b433386bb484 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:37.478269 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f126351-88d2-4f0b-8c3a-b433386bb484 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:37.487832 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f126351-88d2-4f0b-8c3a-b433386bb484 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:37.489349 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f126351-88d2-4f0b-8c3a-b433386bb484 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:37.499021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1423/2833] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:38:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:38:37.780736 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a82b96c-fe14-47e8-8e8d-199597494e56 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:37.782820 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9a82b96c-fe14-47e8-8e8d-199597494e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:37.801548 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a82b96c-fe14-47e8-8e8d-199597494e56 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:37.806108 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b883cb9-bb7b-4c6c-a84a-e49a004d21bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:37.806743 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b883cb9-bb7b-4c6c-a84a-e49a004d21bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:37.807019 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b883cb9-bb7b-4c6c-a84a-e49a004d21bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:37.807289 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b883cb9-bb7b-4c6c-a84a-e49a004d21bd admin admin] PATH_INFO: `/v3/projects/30255cdb6ac84e978bfef02b3af1c3e5` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:37.808665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b883cb9-bb7b-4c6c-a84a-e49a004d21bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=30255cdb6ac84e978bfef02b3af1c3e5)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:37.810573 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b883cb9-bb7b-4c6c-a84a-e49a004d21bd admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:37.859389 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9b883cb9-bb7b-4c6c-a84a-e49a004d21bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30255cdb6ac84e978bfef02b3af1c3e5', 'request_id': 'req-9b883cb9-bb7b-4c6c-a84a-e49a004d21bd'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:37.865019 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9b883cb9-bb7b-4c6c-a84a-e49a004d21bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30255cdb6ac84e978bfef02b3af1c3e5', 'request_id': 'req-9b883cb9-bb7b-4c6c-a84a-e49a004d21bd'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:37.871359 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9b883cb9-bb7b-4c6c-a84a-e49a004d21bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30255cdb6ac84e978bfef02b3af1c3e5', 'request_id': 'req-9b883cb9-bb7b-4c6c-a84a-e49a004d21bd'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:37.886226 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9b883cb9-bb7b-4c6c-a84a-e49a004d21bd admin admin] The token cache is being invalidate because project 30255cdb6ac84e978bfef02b3af1c3e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:38:37.886505 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9b883cb9-bb7b-4c6c-a84a-e49a004d21bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:37.887129 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9b883cb9-bb7b-4c6c-a84a-e49a004d21bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:37.888991 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1411/2834] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:38:37 2022] DELETE /identity/v3/projects/30255cdb6ac84e978bfef02b3af1c3e5 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:38:37.892801 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f126351-88d2-4f0b-8c3a-b433386bb484 service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:37.902851 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f126351-88d2-4f0b-8c3a-b433386bb484 service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:37.921221 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:38:37.962666 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f620bf-8d5f-419b-886a-2db1b3ab11a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:37.963390 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f620bf-8d5f-419b-886a-2db1b3ab11a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:37.963736 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f620bf-8d5f-419b-886a-2db1b3ab11a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:37.964094 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f620bf-8d5f-419b-886a-2db1b3ab11a1 admin admin] PATH_INFO: `/v3/users/77661616f6ff458b8c5a04b07e982d64` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:37.965268 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f620bf-8d5f-419b-886a-2db1b3ab11a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77661616f6ff458b8c5a04b07e982d64)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:37.967275 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f620bf-8d5f-419b-886a-2db1b3ab11a1 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:38.015339 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-79f620bf-8d5f-419b-886a-2db1b3ab11a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77661616f6ff458b8c5a04b07e982d64'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:38.026868 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-79f620bf-8d5f-419b-886a-2db1b3ab11a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77661616f6ff458b8c5a04b07e982d64'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:38.033674 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-79f620bf-8d5f-419b-886a-2db1b3ab11a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77661616f6ff458b8c5a04b07e982d64'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:38.041362 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-79f620bf-8d5f-419b-886a-2db1b3ab11a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77661616f6ff458b8c5a04b07e982d64'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:38.041978 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-79f620bf-8d5f-419b-886a-2db1b3ab11a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77661616f6ff458b8c5a04b07e982d64'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:38.047913 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1424/2835] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:38:37 2022] DELETE /identity/v3/users/77661616f6ff458b8c5a04b07e982d64 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:38:38.310887 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b883cb9-bb7b-4c6c-a84a-e49a004d21bd admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:38.312288 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9b883cb9-bb7b-4c6c-a84a-e49a004d21bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:38.325223 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b883cb9-bb7b-4c6c-a84a-e49a004d21bd admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:38.396618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b6e48-d055-451c-9474-bbf3a4e5bb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:38.397314 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b6e48-d055-451c-9474-bbf3a4e5bb6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:38.397597 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b6e48-d055-451c-9474-bbf3a4e5bb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:38.397870 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b6e48-d055-451c-9474-bbf3a4e5bb6b admin admin] PATH_INFO: `/v3/projects/695915870eb34441bc11328af6af15a5` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:38.399033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b6e48-d055-451c-9474-bbf3a4e5bb6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=695915870eb34441bc11328af6af15a5)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:38.400927 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b6e48-d055-451c-9474-bbf3a4e5bb6b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:38.449793 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-655b6e48-d055-451c-9474-bbf3a4e5bb6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '695915870eb34441bc11328af6af15a5', 'request_id': 'req-655b6e48-d055-451c-9474-bbf3a4e5bb6b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:38.455464 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-655b6e48-d055-451c-9474-bbf3a4e5bb6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '695915870eb34441bc11328af6af15a5', 'request_id': 'req-655b6e48-d055-451c-9474-bbf3a4e5bb6b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:38.456543 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:38:38.463550 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-655b6e48-d055-451c-9474-bbf3a4e5bb6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '695915870eb34441bc11328af6af15a5', 'request_id': 'req-655b6e48-d055-451c-9474-bbf3a4e5bb6b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:38.475704 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-655b6e48-d055-451c-9474-bbf3a4e5bb6b admin admin] The token cache is being invalidate because project 695915870eb34441bc11328af6af15a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:38:38.476098 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-655b6e48-d055-451c-9474-bbf3a4e5bb6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:38.476695 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-655b6e48-d055-451c-9474-bbf3a4e5bb6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:38.478252 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1412/2836] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:38:38 2022] DELETE /identity/v3/projects/695915870eb34441bc11328af6af15a5 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:38:38.483047 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc1639-6d4f-4906-8ad0-295bc50d6ebe None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:38.483337 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc1639-6d4f-4906-8ad0-295bc50d6ebe None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:38.483612 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc1639-6d4f-4906-8ad0-295bc50d6ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:38.503494 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-e4bc1639-6d4f-4906-8ad0-295bc50d6ebe None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:38:38.505172 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-e4bc1639-6d4f-4906-8ad0-295bc50d6ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:38.561656 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1425/2837] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:38:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:38:38.569436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008c869-902b-432c-9e59-a282894311e4 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:38.569724 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008c869-902b-432c-9e59-a282894311e4 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:38.569998 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008c869-902b-432c-9e59-a282894311e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:38.589870 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-1008c869-902b-432c-9e59-a282894311e4 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:38:38.591071 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1008c869-902b-432c-9e59-a282894311e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:38.599547 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1413/2838] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:38:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:38:38.603532 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc1639-6d4f-4906-8ad0-295bc50d6ebe None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:38.613600 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc1639-6d4f-4906-8ad0-295bc50d6ebe None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:38.618119 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cda9f1-9831-431f-9a38-bac3c8b7df47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:38.618706 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cda9f1-9831-431f-9a38-bac3c8b7df47 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:38.618975 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cda9f1-9831-431f-9a38-bac3c8b7df47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:38.619240 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cda9f1-9831-431f-9a38-bac3c8b7df47 admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:38.619871 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cda9f1-9831-431f-9a38-bac3c8b7df47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:38:38.620164 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cda9f1-9831-431f-9a38-bac3c8b7df47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:38.621766 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cda9f1-9831-431f-9a38-bac3c8b7df47 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:38.640473 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1426/2839] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:38:38.645417 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008c869-902b-432c-9e59-a282894311e4 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:38.654674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008c869-902b-432c-9e59-a282894311e4 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:38.659435 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7de2dd-60bc-4d8c-9110-e5d8dc4507e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:38.660104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7de2dd-60bc-4d8c-9110-e5d8dc4507e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:38.660900 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7de2dd-60bc-4d8c-9110-e5d8dc4507e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:38.661175 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7de2dd-60bc-4d8c-9110-e5d8dc4507e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:38.661838 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7de2dd-60bc-4d8c-9110-e5d8dc4507e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:38.663768 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7de2dd-60bc-4d8c-9110-e5d8dc4507e3 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:38.691883 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1414/2840] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:38:38 2022] POST /identity/v3/projects => generated 399 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:38:38.696221 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cda9f1-9831-431f-9a38-bac3c8b7df47 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:38.705886 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cda9f1-9831-431f-9a38-bac3c8b7df47 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:38.732327 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:38:38.763542 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a223145-bd7c-4efe-889b-ea47e797c905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:38.764079 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a223145-bd7c-4efe-889b-ea47e797c905 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:38.764353 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a223145-bd7c-4efe-889b-ea47e797c905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:38.764621 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a223145-bd7c-4efe-889b-ea47e797c905 admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:38.765266 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a223145-bd7c-4efe-889b-ea47e797c905 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:38.767182 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a223145-bd7c-4efe-889b-ea47e797c905 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:38.788559 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1427/2841] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:38:38 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:38:38.792614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7de2dd-60bc-4d8c-9110-e5d8dc4507e3 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:38.802954 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7de2dd-60bc-4d8c-9110-e5d8dc4507e3 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:38.807770 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb070c-d842-436d-b103-c254bd5b0be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:38.808403 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb070c-d842-436d-b103-c254bd5b0be3 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:38.808697 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb070c-d842-436d-b103-c254bd5b0be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:38.808969 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb070c-d842-436d-b103-c254bd5b0be3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:38.809775 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fb070c-d842-436d-b103-c254bd5b0be3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:38.811429 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fb070c-d842-436d-b103-c254bd5b0be3 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:38.823111 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1415/2842] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:38:38.827062 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a223145-bd7c-4efe-889b-ea47e797c905 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:38.836943 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a223145-bd7c-4efe-889b-ea47e797c905 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:38.841886 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89884286-be2f-41fa-b17a-82c5e7fdbef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:38.842483 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89884286-be2f-41fa-b17a-82c5e7fdbef4 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:38.842761 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89884286-be2f-41fa-b17a-82c5e7fdbef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:38.843034 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89884286-be2f-41fa-b17a-82c5e7fdbef4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:38.843672 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89884286-be2f-41fa-b17a-82c5e7fdbef4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:38.845315 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89884286-be2f-41fa-b17a-82c5e7fdbef4 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:38.856475 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1428/2843] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:38:38.860755 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb070c-d842-436d-b103-c254bd5b0be3 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:38.870941 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb070c-d842-436d-b103-c254bd5b0be3 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:38.875516 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a59dea-24d0-4c01-b8e6-7bf21b469b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:38.876153 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a59dea-24d0-4c01-b8e6-7bf21b469b0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:38.876428 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a59dea-24d0-4c01-b8e6-7bf21b469b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:38.876700 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a59dea-24d0-4c01-b8e6-7bf21b469b0b admin admin] PATH_INFO: `/v3/projects/ebb58a8683f7435da2a43a77fa2b1350/users/61082d679149400299502eef48a5d981/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:38.892645 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a59dea-24d0-4c01-b8e6-7bf21b469b0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebb58a8683f7435da2a43a77fa2b1350, user_id=61082d679149400299502eef48a5d981, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:38.896504 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a59dea-24d0-4c01-b8e6-7bf21b469b0b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:38.909286 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1416/2844] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:38:38 2022] PUT /identity/v3/projects/ebb58a8683f7435da2a43a77fa2b1350/users/61082d679149400299502eef48a5d981/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:38:38.971501 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105411a-ca15-43b7-8c3b-6db3fe788de7 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:38.972106 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105411a-ca15-43b7-8c3b-6db3fe788de7 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:38.972106 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105411a-ca15-43b7-8c3b-6db3fe788de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:38.999745 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-a105411a-ca15-43b7-8c3b-6db3fe788de7 None None] MFA Rules not processed for user `61082d679149400299502eef48a5d981`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:38:39.000945 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a105411a-ca15-43b7-8c3b-6db3fe788de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:39.063623 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1429/2845] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:38:38 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:38:39.068588 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a59dea-24d0-4c01-b8e6-7bf21b469b0b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:39.128569 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a59dea-24d0-4c01-b8e6-7bf21b469b0b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:39.133708 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92156994-a1be-4d36-9cfd-3c1f93ddbda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:39.134306 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92156994-a1be-4d36-9cfd-3c1f93ddbda3 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:39.134586 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92156994-a1be-4d36-9cfd-3c1f93ddbda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:39.134859 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92156994-a1be-4d36-9cfd-3c1f93ddbda3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:39.135492 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92156994-a1be-4d36-9cfd-3c1f93ddbda3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:39.137399 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92156994-a1be-4d36-9cfd-3c1f93ddbda3 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:39.163440 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1417/2846] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:38:39 2022] POST /identity/v3/projects => generated 401 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:38:39.167488 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a105411a-ca15-43b7-8c3b-6db3fe788de7 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:39.178010 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a105411a-ca15-43b7-8c3b-6db3fe788de7 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:39.236615 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40328360-1ed2-4bb0-aafe-ceba9391f6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:39.237217 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40328360-1ed2-4bb0-aafe-ceba9391f6e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:39.237522 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40328360-1ed2-4bb0-aafe-ceba9391f6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:39.237787 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40328360-1ed2-4bb0-aafe-ceba9391f6e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:39.238928 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40328360-1ed2-4bb0-aafe-ceba9391f6e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:39.240827 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40328360-1ed2-4bb0-aafe-ceba9391f6e6 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:39.261771 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1430/2847] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:38:39 2022] POST /identity/v3/users => generated 307 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:38:39.265825 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92156994-a1be-4d36-9cfd-3c1f93ddbda3 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:39.276128 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92156994-a1be-4d36-9cfd-3c1f93ddbda3 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:39.280745 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34faa988-e700-4715-ada5-d09c943edf80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:39.281378 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34faa988-e700-4715-ada5-d09c943edf80 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:39.281659 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34faa988-e700-4715-ada5-d09c943edf80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:39.281934 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34faa988-e700-4715-ada5-d09c943edf80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:39.282596 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34faa988-e700-4715-ada5-d09c943edf80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:39.284148 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34faa988-e700-4715-ada5-d09c943edf80 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:39.294585 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1418/2848] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:38:39.298654 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40328360-1ed2-4bb0-aafe-ceba9391f6e6 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:39.309374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40328360-1ed2-4bb0-aafe-ceba9391f6e6 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:39.313855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a64f0-de06-4d77-8618-d9cbd54b3802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:39.314493 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a64f0-de06-4d77-8618-d9cbd54b3802 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:39.314767 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a64f0-de06-4d77-8618-d9cbd54b3802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:39.315041 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a64f0-de06-4d77-8618-d9cbd54b3802 admin admin] PATH_INFO: `/v3/projects/dfc910cd982441e99b769ef3afe4abea/users/628afcdebae543e0943246db951716d9/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:39.331934 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a64f0-de06-4d77-8618-d9cbd54b3802 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfc910cd982441e99b769ef3afe4abea, user_id=628afcdebae543e0943246db951716d9, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:39.334634 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a64f0-de06-4d77-8618-d9cbd54b3802 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:39.347613 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1431/2849] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:38:39 2022] PUT /identity/v3/projects/dfc910cd982441e99b769ef3afe4abea/users/628afcdebae543e0943246db951716d9/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:38:39.351589 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34faa988-e700-4715-ada5-d09c943edf80 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:39.362271 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34faa988-e700-4715-ada5-d09c943edf80 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:39.416117 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:38:39.422753 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf2a65-a2b5-4a10-a99c-ed40d66b1f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:39.423743 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf2a65-a2b5-4a10-a99c-ed40d66b1f58 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:39.424356 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf2a65-a2b5-4a10-a99c-ed40d66b1f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:39.425168 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf2a65-a2b5-4a10-a99c-ed40d66b1f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:39.426356 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acf2a65-a2b5-4a10-a99c-ed40d66b1f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:39.428610 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acf2a65-a2b5-4a10-a99c-ed40d66b1f58 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:39.439323 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1419/2850] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:38:39.443723 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a64f0-de06-4d77-8618-d9cbd54b3802 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:39.449738 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:38:39.454937 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a64f0-de06-4d77-8618-d9cbd54b3802 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:39.461868 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d3884-edec-49a5-b73e-5ed5158011da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:39.462726 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d3884-edec-49a5-b73e-5ed5158011da admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:39.463336 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d3884-edec-49a5-b73e-5ed5158011da admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:39.463613 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d3884-edec-49a5-b73e-5ed5158011da admin admin] PATH_INFO: `/v3/projects/dfc910cd982441e99b769ef3afe4abea/users/628afcdebae543e0943246db951716d9/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:39.466428 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23d3884-edec-49a5-b73e-5ed5158011da admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfc910cd982441e99b769ef3afe4abea, user_id=628afcdebae543e0943246db951716d9, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:39.469120 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23d3884-edec-49a5-b73e-5ed5158011da admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:39.481326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1432/2851] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:38:39 2022] PUT /identity/v3/projects/dfc910cd982441e99b769ef3afe4abea/users/628afcdebae543e0943246db951716d9/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:38:39.492176 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04b579-b8a1-4bb5-9b5c-3353b83ba4f3 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:39.492553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04b579-b8a1-4bb5-9b5c-3353b83ba4f3 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:39.493062 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04b579-b8a1-4bb5-9b5c-3353b83ba4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:39.546463 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-cf04b579-b8a1-4bb5-9b5c-3353b83ba4f3 None None] MFA Rules not processed for user `628afcdebae543e0943246db951716d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:38:39.548212 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-cf04b579-b8a1-4bb5-9b5c-3353b83ba4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:39.640613 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1420/2852] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:38:39 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:38:39.649291 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d3884-edec-49a5-b73e-5ed5158011da admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:39.654563 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a23d3884-edec-49a5-b73e-5ed5158011da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:39.715972 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d3884-edec-49a5-b73e-5ed5158011da admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:39.719694 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d46db-c5ed-49f1-ae80-23553f370242 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:39.720812 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d46db-c5ed-49f1-ae80-23553f370242 service cinder] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:39.721171 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d46db-c5ed-49f1-ae80-23553f370242 service cinder] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:39.721496 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d46db-c5ed-49f1-ae80-23553f370242 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:39.730785 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d46db-c5ed-49f1-ae80-23553f370242 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:39.732204 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d46db-c5ed-49f1-ae80-23553f370242 service cinder] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:39.741659 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1433/2853] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:38:39 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:38:39.824616 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04b579-b8a1-4bb5-9b5c-3353b83ba4f3 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:39.834379 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04b579-b8a1-4bb5-9b5c-3353b83ba4f3 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:39.838266 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e5ab7-00f5-4e5d-a761-e000c5903140 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:39.839087 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e5ab7-00f5-4e5d-a761-e000c5903140 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:39.839542 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e5ab7-00f5-4e5d-a761-e000c5903140 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:39.839993 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e5ab7-00f5-4e5d-a761-e000c5903140 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:39.899918 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e5ab7-00f5-4e5d-a761-e000c5903140 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:39.901443 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e5ab7-00f5-4e5d-a761-e000c5903140 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:39.910504 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1421/2854] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:38:39 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:38:43.998044 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d46db-c5ed-49f1-ae80-23553f370242 service cinder] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:43.998903 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-3d7d46db-c5ed-49f1-ae80-23553f370242 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:44.011951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e5ab7-00f5-4e5d-a761-e000c5903140 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:44.078553 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e5ab7-00f5-4e5d-a761-e000c5903140 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:44.083991 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d46db-c5ed-49f1-ae80-23553f370242 service cinder] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:44.084137 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061c381-d976-432c-8386-90f3a677f59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:44.084747 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061c381-d976-432c-8386-90f3a677f59e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:44.085120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061c381-d976-432c-8386-90f3a677f59e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:44.085509 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061c381-d976-432c-8386-90f3a677f59e admin admin] PATH_INFO: `/v3/users/61082d679149400299502eef48a5d981` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:44.086601 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4061c381-d976-432c-8386-90f3a677f59e admin admin] RBAC: Authorizing `identity:delete_user(user_id=61082d679149400299502eef48a5d981)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:44.088504 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4061c381-d976-432c-8386-90f3a677f59e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:44.089655 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076b0f3-a19d-441e-ac6e-d3c2af7f7b06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:44.090259 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076b0f3-a19d-441e-ac6e-d3c2af7f7b06 service nova] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:44.090535 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076b0f3-a19d-441e-ac6e-d3c2af7f7b06 service nova] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:44.090804 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076b0f3-a19d-441e-ac6e-d3c2af7f7b06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:44.093072 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-1076b0f3-a19d-441e-ac6e-d3c2af7f7b06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:44.106280 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1076b0f3-a19d-441e-ac6e-d3c2af7f7b06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:44.107860 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1076b0f3-a19d-441e-ac6e-d3c2af7f7b06 service nova] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:44.111055 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:38:44.123574 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1434/2855] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:38:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:38:44.145446 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4061c381-d976-432c-8386-90f3a677f59e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61082d679149400299502eef48a5d981'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:44.153351 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4061c381-d976-432c-8386-90f3a677f59e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61082d679149400299502eef48a5d981'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:44.168005 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4061c381-d976-432c-8386-90f3a677f59e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61082d679149400299502eef48a5d981'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:44.177576 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4061c381-d976-432c-8386-90f3a677f59e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61082d679149400299502eef48a5d981'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:44.178356 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4061c381-d976-432c-8386-90f3a677f59e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61082d679149400299502eef48a5d981'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:44.184780 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1422/2856] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:38:44 2022] DELETE /identity/v3/users/61082d679149400299502eef48a5d981 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:38:44.189473 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b11277-771e-491f-9cbc-42096d5ac69a None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:44.189875 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b11277-771e-491f-9cbc-42096d5ac69a None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:44.190219 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b11277-771e-491f-9cbc-42096d5ac69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:44.211880 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-d7b11277-771e-491f-9cbc-42096d5ac69a None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:38:44.213179 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-d7b11277-771e-491f-9cbc-42096d5ac69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:44.281575 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1435/2857] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:38:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:38:44.292970 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061c381-d976-432c-8386-90f3a677f59e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:44.294534 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4061c381-d976-432c-8386-90f3a677f59e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:44.359740 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061c381-d976-432c-8386-90f3a677f59e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:44.364234 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103e3f6-7c10-47b9-8898-eecbc7938118 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:44.364642 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103e3f6-7c10-47b9-8898-eecbc7938118 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:44.364921 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103e3f6-7c10-47b9-8898-eecbc7938118 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:44.365194 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103e3f6-7c10-47b9-8898-eecbc7938118 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:44.377122 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2103e3f6-7c10-47b9-8898-eecbc7938118 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:44.377345 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2103e3f6-7c10-47b9-8898-eecbc7938118 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:44.388890 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1423/2858] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:38:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:38:44.810675 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b11277-771e-491f-9cbc-42096d5ac69a None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:44.813516 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-d7b11277-771e-491f-9cbc-42096d5ac69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:44.835077 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b11277-771e-491f-9cbc-42096d5ac69a None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:44.858987 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c245eadf-f5b7-4525-9ec7-075d2694a59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:44.860024 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245eadf-f5b7-4525-9ec7-075d2694a59e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:44.860328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245eadf-f5b7-4525-9ec7-075d2694a59e admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:44.860605 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245eadf-f5b7-4525-9ec7-075d2694a59e admin admin] PATH_INFO: `/v3/projects/ebb58a8683f7435da2a43a77fa2b1350` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:44.861773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c245eadf-f5b7-4525-9ec7-075d2694a59e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebb58a8683f7435da2a43a77fa2b1350)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:44.864310 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c245eadf-f5b7-4525-9ec7-075d2694a59e admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:44.876734 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:38:44.916484 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-c245eadf-f5b7-4525-9ec7-075d2694a59e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebb58a8683f7435da2a43a77fa2b1350', 'request_id': 'req-c245eadf-f5b7-4525-9ec7-075d2694a59e'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:44.924374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-c245eadf-f5b7-4525-9ec7-075d2694a59e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebb58a8683f7435da2a43a77fa2b1350', 'request_id': 'req-c245eadf-f5b7-4525-9ec7-075d2694a59e'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:44.930660 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-c245eadf-f5b7-4525-9ec7-075d2694a59e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebb58a8683f7435da2a43a77fa2b1350', 'request_id': 'req-c245eadf-f5b7-4525-9ec7-075d2694a59e'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:44.943005 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-c245eadf-f5b7-4525-9ec7-075d2694a59e admin admin] The token cache is being invalidate because project ebb58a8683f7435da2a43a77fa2b1350 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:38:44.943313 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-c245eadf-f5b7-4525-9ec7-075d2694a59e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:44.943977 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-c245eadf-f5b7-4525-9ec7-075d2694a59e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:44.945482 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1436/2859] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:38:44 2022] DELETE /identity/v3/projects/ebb58a8683f7435da2a43a77fa2b1350 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:38:44.949545 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103e3f6-7c10-47b9-8898-eecbc7938118 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:44.962279 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103e3f6-7c10-47b9-8898-eecbc7938118 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:45.022965 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03228ecb-2b48-4af4-9b87-a530f0a256e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:45.023975 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03228ecb-2b48-4af4-9b87-a530f0a256e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:45.023975 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03228ecb-2b48-4af4-9b87-a530f0a256e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:45.024179 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03228ecb-2b48-4af4-9b87-a530f0a256e1 admin admin] PATH_INFO: `/v3/users/628afcdebae543e0943246db951716d9` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:45.025361 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03228ecb-2b48-4af4-9b87-a530f0a256e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=628afcdebae543e0943246db951716d9)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:45.027284 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03228ecb-2b48-4af4-9b87-a530f0a256e1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:45.069999 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:38:45.077324 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-03228ecb-2b48-4af4-9b87-a530f0a256e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '628afcdebae543e0943246db951716d9'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:45.085075 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-03228ecb-2b48-4af4-9b87-a530f0a256e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '628afcdebae543e0943246db951716d9'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:45.098728 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-03228ecb-2b48-4af4-9b87-a530f0a256e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '628afcdebae543e0943246db951716d9'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:45.105850 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-03228ecb-2b48-4af4-9b87-a530f0a256e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '628afcdebae543e0943246db951716d9'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:45.106446 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-03228ecb-2b48-4af4-9b87-a530f0a256e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '628afcdebae543e0943246db951716d9'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:45.112595 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1424/2860] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:38:44 2022] DELETE /identity/v3/users/628afcdebae543e0943246db951716d9 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:38:45.381017 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c245eadf-f5b7-4525-9ec7-075d2694a59e admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:45.382529 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-c245eadf-f5b7-4525-9ec7-075d2694a59e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:45.399856 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c245eadf-f5b7-4525-9ec7-075d2694a59e admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:45.456634 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e6c835-d8b9-4401-b830-5ffcfbf3731b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:45.457398 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e6c835-d8b9-4401-b830-5ffcfbf3731b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:45.457676 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e6c835-d8b9-4401-b830-5ffcfbf3731b admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:45.457945 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e6c835-d8b9-4401-b830-5ffcfbf3731b admin admin] PATH_INFO: `/v3/projects/dfc910cd982441e99b769ef3afe4abea` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:45.459040 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e6c835-d8b9-4401-b830-5ffcfbf3731b admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfc910cd982441e99b769ef3afe4abea)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:45.460928 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e6c835-d8b9-4401-b830-5ffcfbf3731b admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:45.508230 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-69e6c835-d8b9-4401-b830-5ffcfbf3731b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfc910cd982441e99b769ef3afe4abea', 'request_id': 'req-69e6c835-d8b9-4401-b830-5ffcfbf3731b'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:45.515424 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-69e6c835-d8b9-4401-b830-5ffcfbf3731b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfc910cd982441e99b769ef3afe4abea', 'request_id': 'req-69e6c835-d8b9-4401-b830-5ffcfbf3731b'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:45.521004 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-69e6c835-d8b9-4401-b830-5ffcfbf3731b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfc910cd982441e99b769ef3afe4abea', 'request_id': 'req-69e6c835-d8b9-4401-b830-5ffcfbf3731b'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:45.532038 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-69e6c835-d8b9-4401-b830-5ffcfbf3731b admin admin] The token cache is being invalidate because project dfc910cd982441e99b769ef3afe4abea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:38:45.532328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-69e6c835-d8b9-4401-b830-5ffcfbf3731b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:45.532951 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-69e6c835-d8b9-4401-b830-5ffcfbf3731b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:45.534416 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1437/2861] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:38:45 2022] DELETE /identity/v3/projects/dfc910cd982441e99b769ef3afe4abea => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:38:45.539376 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1d508-f049-45d6-848c-65de26d8c2b5 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:45.539666 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1d508-f049-45d6-848c-65de26d8c2b5 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:45.539939 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1d508-f049-45d6-848c-65de26d8c2b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:45.559490 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-99c1d508-f049-45d6-848c-65de26d8c2b5 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:38:45.560675 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-99c1d508-f049-45d6-848c-65de26d8c2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:45.619614 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1425/2862] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:38:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:38:45.628636 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb952c-3645-45d1-afde-782ca3ee5808 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:45.628878 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb952c-3645-45d1-afde-782ca3ee5808 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:45.629157 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb952c-3645-45d1-afde-782ca3ee5808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:45.648360 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-f8cb952c-3645-45d1-afde-782ca3ee5808 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:38:45.649589 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-f8cb952c-3645-45d1-afde-782ca3ee5808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:45.657499 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1438/2863] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:38:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:38:45.662225 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1d508-f049-45d6-848c-65de26d8c2b5 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:45.672083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1d508-f049-45d6-848c-65de26d8c2b5 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:45.677274 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca36c32a-b533-455d-b58a-e6a10634a47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:45.678522 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36c32a-b533-455d-b58a-e6a10634a47c admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:45.678832 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36c32a-b533-455d-b58a-e6a10634a47c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:45.679109 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36c32a-b533-455d-b58a-e6a10634a47c admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:45.679758 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca36c32a-b533-455d-b58a-e6a10634a47c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:38:45.680060 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca36c32a-b533-455d-b58a-e6a10634a47c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:45.681671 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca36c32a-b533-455d-b58a-e6a10634a47c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:45.693131 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1426/2864] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:38:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:38:45.697557 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb952c-3645-45d1-afde-782ca3ee5808 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:45.706759 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb952c-3645-45d1-afde-782ca3ee5808 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:45.711406 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1994c-17b6-4a04-9d0e-224a50acbf2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:45.711996 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1994c-17b6-4a04-9d0e-224a50acbf2c admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:45.712276 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1994c-17b6-4a04-9d0e-224a50acbf2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:45.712545 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1994c-17b6-4a04-9d0e-224a50acbf2c admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:45.713162 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1994c-17b6-4a04-9d0e-224a50acbf2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:45.715079 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1994c-17b6-4a04-9d0e-224a50acbf2c admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:45.742971 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1439/2865] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:38:45 2022] POST /identity/v3/projects => generated 389 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:38:45.747188 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca36c32a-b533-455d-b58a-e6a10634a47c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:45.757209 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca36c32a-b533-455d-b58a-e6a10634a47c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:45.812524 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe73c6-1a60-4bef-81f7-69b00cf54e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:45.813125 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe73c6-1a60-4bef-81f7-69b00cf54e8e admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:45.813441 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe73c6-1a60-4bef-81f7-69b00cf54e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:45.813718 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe73c6-1a60-4bef-81f7-69b00cf54e8e admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:45.814342 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe73c6-1a60-4bef-81f7-69b00cf54e8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:45.816270 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe73c6-1a60-4bef-81f7-69b00cf54e8e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:45.837154 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1427/2866] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:38:45 2022] POST /identity/v3/users => generated 295 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:38:45.841488 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1994c-17b6-4a04-9d0e-224a50acbf2c admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:45.846900 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:38:45.853326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1994c-17b6-4a04-9d0e-224a50acbf2c admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:45.858862 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9981f680-24c8-4e25-b011-f0eb5abd1b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:45.859741 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981f680-24c8-4e25-b011-f0eb5abd1b0b admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:45.860043 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981f680-24c8-4e25-b011-f0eb5abd1b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:45.860315 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981f680-24c8-4e25-b011-f0eb5abd1b0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:45.860949 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9981f680-24c8-4e25-b011-f0eb5abd1b0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:45.862563 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9981f680-24c8-4e25-b011-f0eb5abd1b0b admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:45.872461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1440/2867] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:38:45.876341 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe73c6-1a60-4bef-81f7-69b00cf54e8e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:45.886769 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe73c6-1a60-4bef-81f7-69b00cf54e8e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:45.891410 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbbefde-1859-439b-80e0-94f046466a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:45.892001 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbefde-1859-439b-80e0-94f046466a7c admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:45.892283 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbefde-1859-439b-80e0-94f046466a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:45.892556 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbefde-1859-439b-80e0-94f046466a7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:45.893192 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbbefde-1859-439b-80e0-94f046466a7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:45.895352 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbbefde-1859-439b-80e0-94f046466a7c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:45.905791 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1428/2868] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:38:45.909723 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9981f680-24c8-4e25-b011-f0eb5abd1b0b admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:45.919482 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9981f680-24c8-4e25-b011-f0eb5abd1b0b admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:45.924006 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c26a39-4862-495b-ac1c-439eff51fb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:45.924649 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c26a39-4862-495b-ac1c-439eff51fb14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:45.924927 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c26a39-4862-495b-ac1c-439eff51fb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:45.925202 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c26a39-4862-495b-ac1c-439eff51fb14 admin admin] PATH_INFO: `/v3/projects/1e5cbd1c12c14c64aa47d1543f217fed/users/37393e87360e40278950eaac769bb844/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:45.939722 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c26a39-4862-495b-ac1c-439eff51fb14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e5cbd1c12c14c64aa47d1543f217fed, user_id=37393e87360e40278950eaac769bb844, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:45.943247 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c26a39-4862-495b-ac1c-439eff51fb14 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:45.955431 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1441/2869] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:38:45 2022] PUT /identity/v3/projects/1e5cbd1c12c14c64aa47d1543f217fed/users/37393e87360e40278950eaac769bb844/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:38:45.966344 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75d428-694e-442c-89f5-e9a686a1b405 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:45.966628 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75d428-694e-442c-89f5-e9a686a1b405 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:45.966903 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75d428-694e-442c-89f5-e9a686a1b405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:45.996221 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-fd75d428-694e-442c-89f5-e9a686a1b405 None None] MFA Rules not processed for user `37393e87360e40278950eaac769bb844`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:38:45.997489 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-fd75d428-694e-442c-89f5-e9a686a1b405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:46.010277 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:38:46.064621 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1429/2870] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:38:45 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:38:46.069858 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c26a39-4862-495b-ac1c-439eff51fb14 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:46.130306 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c26a39-4862-495b-ac1c-439eff51fb14 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:46.134926 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0022ba4-1d3b-4426-91e4-9bd5d924a025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:46.135528 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0022ba4-1d3b-4426-91e4-9bd5d924a025 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:46.135809 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0022ba4-1d3b-4426-91e4-9bd5d924a025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:46.136087 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0022ba4-1d3b-4426-91e4-9bd5d924a025 admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:46.136702 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0022ba4-1d3b-4426-91e4-9bd5d924a025 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:46.138678 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0022ba4-1d3b-4426-91e4-9bd5d924a025 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:46.164817 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1442/2871] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:38:46 2022] POST /identity/v3/projects => generated 387 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:38:46.168907 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd75d428-694e-442c-89f5-e9a686a1b405 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:46.180895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd75d428-694e-442c-89f5-e9a686a1b405 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:46.236478 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d74f1-bb93-4ecc-9fa8-09ead8a7fb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:46.237080 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d74f1-bb93-4ecc-9fa8-09ead8a7fb99 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:46.237390 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d74f1-bb93-4ecc-9fa8-09ead8a7fb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:46.237665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d74f1-bb93-4ecc-9fa8-09ead8a7fb99 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:46.238783 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949d74f1-bb93-4ecc-9fa8-09ead8a7fb99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:46.240684 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949d74f1-bb93-4ecc-9fa8-09ead8a7fb99 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:46.261376 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1430/2872] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:38:46 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:38:46.265303 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0022ba4-1d3b-4426-91e4-9bd5d924a025 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:46.275535 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0022ba4-1d3b-4426-91e4-9bd5d924a025 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:46.280082 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10fe06-b54a-4344-8c4d-1da5eae7ff42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:46.280674 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10fe06-b54a-4344-8c4d-1da5eae7ff42 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:46.280952 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10fe06-b54a-4344-8c4d-1da5eae7ff42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:46.281221 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10fe06-b54a-4344-8c4d-1da5eae7ff42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:46.281918 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e10fe06-b54a-4344-8c4d-1da5eae7ff42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:46.283514 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e10fe06-b54a-4344-8c4d-1da5eae7ff42 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:46.294066 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1443/2873] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:38:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:38:46.298017 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d74f1-bb93-4ecc-9fa8-09ead8a7fb99 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:46.308474 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d74f1-bb93-4ecc-9fa8-09ead8a7fb99 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:46.313081 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8284e304-221f-41c2-9560-b09028777671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:46.313770 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284e304-221f-41c2-9560-b09028777671 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:46.314034 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284e304-221f-41c2-9560-b09028777671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:46.314308 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284e304-221f-41c2-9560-b09028777671 admin admin] PATH_INFO: `/v3/projects/a7c51d0565184eab93c3929586b1dbc6/users/4a75fd1d01254f1ca942078be67ecfaf/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:46.329083 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8284e304-221f-41c2-9560-b09028777671 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7c51d0565184eab93c3929586b1dbc6, user_id=4a75fd1d01254f1ca942078be67ecfaf, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:46.331753 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8284e304-221f-41c2-9560-b09028777671 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:46.343987 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1431/2874] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:38:46 2022] PUT /identity/v3/projects/a7c51d0565184eab93c3929586b1dbc6/users/4a75fd1d01254f1ca942078be67ecfaf/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:38:46.348288 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10fe06-b54a-4344-8c4d-1da5eae7ff42 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:46.357953 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10fe06-b54a-4344-8c4d-1da5eae7ff42 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:46.414317 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3279f16-fab2-4f33-b7df-c77cf66ff1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:46.414970 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3279f16-fab2-4f33-b7df-c77cf66ff1ce admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:46.415145 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3279f16-fab2-4f33-b7df-c77cf66ff1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:46.415419 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3279f16-fab2-4f33-b7df-c77cf66ff1ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:46.416056 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3279f16-fab2-4f33-b7df-c77cf66ff1ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:46.417695 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3279f16-fab2-4f33-b7df-c77cf66ff1ce admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:46.431748 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1444/2875] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:38:46 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:38:46.438762 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8284e304-221f-41c2-9560-b09028777671 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:46.449521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8284e304-221f-41c2-9560-b09028777671 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:46.454109 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34c058-1096-4e55-88b8-6452aad634a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:46.454755 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34c058-1096-4e55-88b8-6452aad634a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:46.455040 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34c058-1096-4e55-88b8-6452aad634a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:46.455317 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34c058-1096-4e55-88b8-6452aad634a7 admin admin] PATH_INFO: `/v3/projects/a7c51d0565184eab93c3929586b1dbc6/users/4a75fd1d01254f1ca942078be67ecfaf/roles/9121ba25a6d84756990f0c3a19e9fecb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:46.457223 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34c058-1096-4e55-88b8-6452aad634a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7c51d0565184eab93c3929586b1dbc6, user_id=4a75fd1d01254f1ca942078be67ecfaf, role_id=9121ba25a6d84756990f0c3a19e9fecb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:46.459866 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34c058-1096-4e55-88b8-6452aad634a7 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:46.469816 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1432/2876] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:38:46 2022] PUT /identity/v3/projects/a7c51d0565184eab93c3929586b1dbc6/users/4a75fd1d01254f1ca942078be67ecfaf/roles/9121ba25a6d84756990f0c3a19e9fecb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:38:46.480398 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03733272-24c1-4270-a71e-9624f621b51b None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:46.480681 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03733272-24c1-4270-a71e-9624f621b51b None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:46.480955 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03733272-24c1-4270-a71e-9624f621b51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:46.509406 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-03733272-24c1-4270-a71e-9624f621b51b None None] MFA Rules not processed for user `4a75fd1d01254f1ca942078be67ecfaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:38:46.510600 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-03733272-24c1-4270-a71e-9624f621b51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:46.536042 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:38:46.577069 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1445/2877] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:38:46 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:38:46.587604 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34c058-1096-4e55-88b8-6452aad634a7 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:46.589354 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7e34c058-1096-4e55-88b8-6452aad634a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:46.646603 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34c058-1096-4e55-88b8-6452aad634a7 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:46.650277 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8481237b-ee53-4d86-ab91-9e694fa767d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:46.650876 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8481237b-ee53-4d86-ab91-9e694fa767d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:46.651155 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8481237b-ee53-4d86-ab91-9e694fa767d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:46.651426 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8481237b-ee53-4d86-ab91-9e694fa767d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:46.661073 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8481237b-ee53-4d86-ab91-9e694fa767d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:46.662536 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8481237b-ee53-4d86-ab91-9e694fa767d5 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:46.672134 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1433/2878] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:38:46 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:38:46.806410 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03733272-24c1-4270-a71e-9624f621b51b None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:46.867374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03733272-24c1-4270-a71e-9624f621b51b None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:46.871867 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7996b0-3052-48c5-b498-5dc580642952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:46.872494 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7996b0-3052-48c5-b498-5dc580642952 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:46.872776 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7996b0-3052-48c5-b498-5dc580642952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:46.873050 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7996b0-3052-48c5-b498-5dc580642952 admin admin] PATH_INFO: `/v3/users/37393e87360e40278950eaac769bb844` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:46.874214 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7996b0-3052-48c5-b498-5dc580642952 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37393e87360e40278950eaac769bb844)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:46.876149 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7996b0-3052-48c5-b498-5dc580642952 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:46.920586 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-4e7996b0-3052-48c5-b498-5dc580642952 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37393e87360e40278950eaac769bb844'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:46.932563 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-4e7996b0-3052-48c5-b498-5dc580642952 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37393e87360e40278950eaac769bb844'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:46.939357 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-4e7996b0-3052-48c5-b498-5dc580642952 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37393e87360e40278950eaac769bb844'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:46.946710 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-4e7996b0-3052-48c5-b498-5dc580642952 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37393e87360e40278950eaac769bb844'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:46.947328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-4e7996b0-3052-48c5-b498-5dc580642952 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37393e87360e40278950eaac769bb844'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:46.953203 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1446/2879] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:38:46 2022] DELETE /identity/v3/users/37393e87360e40278950eaac769bb844 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:38:46.958372 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b010f9-6ed3-455d-bc44-b72527e19ae3 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:46.958664 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b010f9-6ed3-455d-bc44-b72527e19ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:46.958974 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b010f9-6ed3-455d-bc44-b72527e19ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:46.978065 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e3b010f9-6ed3-455d-bc44-b72527e19ae3 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:38:46.979616 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e3b010f9-6ed3-455d-bc44-b72527e19ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:46.983503 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:38:47.040107 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1434/2880] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:38:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:38:47.050006 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7996b0-3052-48c5-b498-5dc580642952 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:47.051727 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-4e7996b0-3052-48c5-b498-5dc580642952 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:47.097551 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:38:47.110439 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7996b0-3052-48c5-b498-5dc580642952 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:47.114126 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b377b83-c6af-4aa6-ac67-358645e11671 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:47.114721 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b377b83-c6af-4aa6-ac67-358645e11671 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:47.114997 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b377b83-c6af-4aa6-ac67-358645e11671 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:47.115265 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b377b83-c6af-4aa6-ac67-358645e11671 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:47.124016 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b377b83-c6af-4aa6-ac67-358645e11671 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:47.125467 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b377b83-c6af-4aa6-ac67-358645e11671 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:47.135443 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1447/2881] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:38:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:38:47.393960 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b010f9-6ed3-455d-bc44-b72527e19ae3 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:47.395416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e3b010f9-6ed3-455d-bc44-b72527e19ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:47.407449 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b010f9-6ed3-455d-bc44-b72527e19ae3 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:47.413002 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416258c-24d7-485e-93fa-fdfe63235b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:47.413674 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416258c-24d7-485e-93fa-fdfe63235b8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:47.413957 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416258c-24d7-485e-93fa-fdfe63235b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:47.414234 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416258c-24d7-485e-93fa-fdfe63235b8f admin admin] PATH_INFO: `/v3/projects/1e5cbd1c12c14c64aa47d1543f217fed` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:47.415433 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3416258c-24d7-485e-93fa-fdfe63235b8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e5cbd1c12c14c64aa47d1543f217fed)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:47.417355 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3416258c-24d7-485e-93fa-fdfe63235b8f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:47.464713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3416258c-24d7-485e-93fa-fdfe63235b8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e5cbd1c12c14c64aa47d1543f217fed', 'request_id': 'req-3416258c-24d7-485e-93fa-fdfe63235b8f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:47.470385 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3416258c-24d7-485e-93fa-fdfe63235b8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e5cbd1c12c14c64aa47d1543f217fed', 'request_id': 'req-3416258c-24d7-485e-93fa-fdfe63235b8f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:47.477267 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3416258c-24d7-485e-93fa-fdfe63235b8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e5cbd1c12c14c64aa47d1543f217fed', 'request_id': 'req-3416258c-24d7-485e-93fa-fdfe63235b8f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:47.489576 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3416258c-24d7-485e-93fa-fdfe63235b8f admin admin] The token cache is being invalidate because project 1e5cbd1c12c14c64aa47d1543f217fed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:38:47.489847 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3416258c-24d7-485e-93fa-fdfe63235b8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:47.490457 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3416258c-24d7-485e-93fa-fdfe63235b8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:47.491929 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1435/2882] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:38:47 2022] DELETE /identity/v3/projects/1e5cbd1c12c14c64aa47d1543f217fed => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:38:47.495916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b377b83-c6af-4aa6-ac67-358645e11671 service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:47.505815 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b377b83-c6af-4aa6-ac67-358645e11671 service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:47.560648 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87e79fd-182e-4a1f-a9ff-e4ca08c8299b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:47.561301 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e79fd-182e-4a1f-a9ff-e4ca08c8299b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:47.561580 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e79fd-182e-4a1f-a9ff-e4ca08c8299b admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:47.561855 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e79fd-182e-4a1f-a9ff-e4ca08c8299b admin admin] PATH_INFO: `/v3/users/4a75fd1d01254f1ca942078be67ecfaf` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:47.562966 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87e79fd-182e-4a1f-a9ff-e4ca08c8299b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a75fd1d01254f1ca942078be67ecfaf)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:47.565217 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87e79fd-182e-4a1f-a9ff-e4ca08c8299b admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:47.612109 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-a87e79fd-182e-4a1f-a9ff-e4ca08c8299b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a75fd1d01254f1ca942078be67ecfaf'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:47.623699 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-a87e79fd-182e-4a1f-a9ff-e4ca08c8299b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a75fd1d01254f1ca942078be67ecfaf'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:47.630478 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-a87e79fd-182e-4a1f-a9ff-e4ca08c8299b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a75fd1d01254f1ca942078be67ecfaf'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:47.637571 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-a87e79fd-182e-4a1f-a9ff-e4ca08c8299b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a75fd1d01254f1ca942078be67ecfaf'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:47.638180 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-a87e79fd-182e-4a1f-a9ff-e4ca08c8299b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a75fd1d01254f1ca942078be67ecfaf'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:47.643986 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1448/2883] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:38:47 2022] DELETE /identity/v3/users/4a75fd1d01254f1ca942078be67ecfaf => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:38:47.915147 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416258c-24d7-485e-93fa-fdfe63235b8f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:47.916911 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3416258c-24d7-485e-93fa-fdfe63235b8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:47.932148 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416258c-24d7-485e-93fa-fdfe63235b8f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:47.989610 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb6574-a748-4dde-b4b7-86bcb888a6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:47.990186 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb6574-a748-4dde-b4b7-86bcb888a6ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:47.990464 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb6574-a748-4dde-b4b7-86bcb888a6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:47.990743 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb6574-a748-4dde-b4b7-86bcb888a6ee admin admin] PATH_INFO: `/v3/projects/a7c51d0565184eab93c3929586b1dbc6` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:47.991838 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cb6574-a748-4dde-b4b7-86bcb888a6ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7c51d0565184eab93c3929586b1dbc6)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:47.993756 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cb6574-a748-4dde-b4b7-86bcb888a6ee admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:48.031936 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:38:48.043488 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c7cb6574-a748-4dde-b4b7-86bcb888a6ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7c51d0565184eab93c3929586b1dbc6', 'request_id': 'req-c7cb6574-a748-4dde-b4b7-86bcb888a6ee'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:48.049047 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c7cb6574-a748-4dde-b4b7-86bcb888a6ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7c51d0565184eab93c3929586b1dbc6', 'request_id': 'req-c7cb6574-a748-4dde-b4b7-86bcb888a6ee'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:48.055248 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c7cb6574-a748-4dde-b4b7-86bcb888a6ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7c51d0565184eab93c3929586b1dbc6', 'request_id': 'req-c7cb6574-a748-4dde-b4b7-86bcb888a6ee'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:48.067605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c7cb6574-a748-4dde-b4b7-86bcb888a6ee admin admin] The token cache is being invalidate because project a7c51d0565184eab93c3929586b1dbc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:38:48.067919 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c7cb6574-a748-4dde-b4b7-86bcb888a6ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:48.068531 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c7cb6574-a748-4dde-b4b7-86bcb888a6ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:48.070119 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1436/2884] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:38:47 2022] DELETE /identity/v3/projects/a7c51d0565184eab93c3929586b1dbc6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:38:48.074927 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469550c8-fa1c-458a-97ca-836f4310cf56 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:48.075215 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469550c8-fa1c-458a-97ca-836f4310cf56 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:48.075540 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469550c8-fa1c-458a-97ca-836f4310cf56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:48.095333 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-469550c8-fa1c-458a-97ca-836f4310cf56 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:38:48.096526 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-469550c8-fa1c-458a-97ca-836f4310cf56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:48.159731 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1449/2885] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:38:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:38:48.171009 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f7f12-c9d2-4e99-8adb-56a55691e16c None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:48.171295 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f7f12-c9d2-4e99-8adb-56a55691e16c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:48.171575 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f7f12-c9d2-4e99-8adb-56a55691e16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:48.191696 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e74f7f12-c9d2-4e99-8adb-56a55691e16c None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:38:48.192982 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e74f7f12-c9d2-4e99-8adb-56a55691e16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:48.200864 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1437/2886] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:38:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:38:48.205066 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469550c8-fa1c-458a-97ca-836f4310cf56 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:48.211781 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:38:48.215831 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469550c8-fa1c-458a-97ca-836f4310cf56 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:48.221962 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29caabf2-b4e2-4fce-8fa2-5300922ae649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:48.222763 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29caabf2-b4e2-4fce-8fa2-5300922ae649 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:48.223211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29caabf2-b4e2-4fce-8fa2-5300922ae649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:48.223644 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29caabf2-b4e2-4fce-8fa2-5300922ae649 admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:48.224291 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29caabf2-b4e2-4fce-8fa2-5300922ae649 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:38:48.224593 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29caabf2-b4e2-4fce-8fa2-5300922ae649 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:48.226220 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29caabf2-b4e2-4fce-8fa2-5300922ae649 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:48.237946 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1450/2887] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:38:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:38:48.242707 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f7f12-c9d2-4e99-8adb-56a55691e16c None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:48.251694 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f7f12-c9d2-4e99-8adb-56a55691e16c None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:48.256991 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b815e447-1125-43bd-a2f7-9609cd4b3967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:48.257623 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815e447-1125-43bd-a2f7-9609cd4b3967 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:48.257907 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815e447-1125-43bd-a2f7-9609cd4b3967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:48.258183 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815e447-1125-43bd-a2f7-9609cd4b3967 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:48.258818 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b815e447-1125-43bd-a2f7-9609cd4b3967 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:48.260700 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b815e447-1125-43bd-a2f7-9609cd4b3967 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:48.287001 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1438/2888] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:38:48 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:38:48.290799 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29caabf2-b4e2-4fce-8fa2-5300922ae649 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:48.300461 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29caabf2-b4e2-4fce-8fa2-5300922ae649 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:48.355197 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217a7f6-2d28-4cd3-b9bb-97c3b91ef103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:48.355787 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217a7f6-2d28-4cd3-b9bb-97c3b91ef103 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:48.356067 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217a7f6-2d28-4cd3-b9bb-97c3b91ef103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:48.356336 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217a7f6-2d28-4cd3-b9bb-97c3b91ef103 admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:48.356956 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b217a7f6-2d28-4cd3-b9bb-97c3b91ef103 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:48.359070 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b217a7f6-2d28-4cd3-b9bb-97c3b91ef103 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:48.380872 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1451/2889] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:38:48 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:38:48.384909 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b815e447-1125-43bd-a2f7-9609cd4b3967 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:48.395027 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b815e447-1125-43bd-a2f7-9609cd4b3967 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:48.399665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51adf098-2291-4226-888c-e023966e42f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:48.400259 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51adf098-2291-4226-888c-e023966e42f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:48.400542 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51adf098-2291-4226-888c-e023966e42f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:48.400818 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51adf098-2291-4226-888c-e023966e42f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:48.401521 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51adf098-2291-4226-888c-e023966e42f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:48.404316 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51adf098-2291-4226-888c-e023966e42f6 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:48.414125 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1439/2890] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:38:48.418170 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217a7f6-2d28-4cd3-b9bb-97c3b91ef103 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:48.428467 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217a7f6-2d28-4cd3-b9bb-97c3b91ef103 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:48.433343 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa07d5-0d76-44c0-9d7f-f692fa4edb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:48.433941 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa07d5-0d76-44c0-9d7f-f692fa4edb14 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:48.434217 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa07d5-0d76-44c0-9d7f-f692fa4edb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:48.434488 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa07d5-0d76-44c0-9d7f-f692fa4edb14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:48.435147 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa07d5-0d76-44c0-9d7f-f692fa4edb14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:48.436853 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa07d5-0d76-44c0-9d7f-f692fa4edb14 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:48.447832 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1452/2891] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:38:48.451746 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51adf098-2291-4226-888c-e023966e42f6 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:48.461815 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51adf098-2291-4226-888c-e023966e42f6 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:48.466596 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23124fa7-2f29-4794-8cdf-282d5a7f2dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:48.467245 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23124fa7-2f29-4794-8cdf-282d5a7f2dd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:48.467522 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23124fa7-2f29-4794-8cdf-282d5a7f2dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:48.467821 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23124fa7-2f29-4794-8cdf-282d5a7f2dd1 admin admin] PATH_INFO: `/v3/projects/4226079fb92c49e39d50cd567d438cae/users/de47e69072d24f38a878a63239673357/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:48.482335 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23124fa7-2f29-4794-8cdf-282d5a7f2dd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4226079fb92c49e39d50cd567d438cae, user_id=de47e69072d24f38a878a63239673357, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:48.485466 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23124fa7-2f29-4794-8cdf-282d5a7f2dd1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:48.498592 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1440/2892] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:38:48 2022] PUT /identity/v3/projects/4226079fb92c49e39d50cd567d438cae/users/de47e69072d24f38a878a63239673357/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:38:48.509492 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52834246-eada-46db-8419-0fe63081fe25 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:48.509996 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52834246-eada-46db-8419-0fe63081fe25 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:48.510097 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52834246-eada-46db-8419-0fe63081fe25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:48.536899 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-52834246-eada-46db-8419-0fe63081fe25 None None] MFA Rules not processed for user `de47e69072d24f38a878a63239673357`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:38:48.538118 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-52834246-eada-46db-8419-0fe63081fe25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:48.601566 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1453/2893] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:38:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:38:48.614277 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23124fa7-2f29-4794-8cdf-282d5a7f2dd1 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:48.615987 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-23124fa7-2f29-4794-8cdf-282d5a7f2dd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:48.673493 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23124fa7-2f29-4794-8cdf-282d5a7f2dd1 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:48.677152 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c67fd-008b-48dd-b743-601732e00abb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:48.677792 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c67fd-008b-48dd-b743-601732e00abb service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:48.678074 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c67fd-008b-48dd-b743-601732e00abb service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:48.678349 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c67fd-008b-48dd-b743-601732e00abb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:48.686713 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c67fd-008b-48dd-b743-601732e00abb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:48.688128 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c67fd-008b-48dd-b743-601732e00abb service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:48.697582 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1441/2894] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:38:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:38:51.250274 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52834246-eada-46db-8419-0fe63081fe25 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:51.310681 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52834246-eada-46db-8419-0fe63081fe25 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:51.315146 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e730d00-225f-41fe-84b1-01bdecdd5152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:51.315762 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e730d00-225f-41fe-84b1-01bdecdd5152 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:51.316039 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e730d00-225f-41fe-84b1-01bdecdd5152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:51.316308 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e730d00-225f-41fe-84b1-01bdecdd5152 admin admin] PATH_INFO: `/v3/users/de47e69072d24f38a878a63239673357` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:51.317426 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e730d00-225f-41fe-84b1-01bdecdd5152 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de47e69072d24f38a878a63239673357)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:51.319325 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e730d00-225f-41fe-84b1-01bdecdd5152 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:51.320754 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:38:51.366015 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6e730d00-225f-41fe-84b1-01bdecdd5152 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de47e69072d24f38a878a63239673357'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:51.377397 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6e730d00-225f-41fe-84b1-01bdecdd5152 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de47e69072d24f38a878a63239673357'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:51.384021 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6e730d00-225f-41fe-84b1-01bdecdd5152 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de47e69072d24f38a878a63239673357'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:51.391019 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6e730d00-225f-41fe-84b1-01bdecdd5152 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de47e69072d24f38a878a63239673357'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:51.391752 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-6e730d00-225f-41fe-84b1-01bdecdd5152 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de47e69072d24f38a878a63239673357'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:51.398146 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1454/2895] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:38:51 2022] DELETE /identity/v3/users/de47e69072d24f38a878a63239673357 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:38:51.402630 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c6905-51ed-4b08-8515-2580e701ccb7 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:51.403153 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c6905-51ed-4b08-8515-2580e701ccb7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:51.403153 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c6905-51ed-4b08-8515-2580e701ccb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:51.422625 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-1c6c6905-51ed-4b08-8515-2580e701ccb7 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:38:51.423861 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1c6c6905-51ed-4b08-8515-2580e701ccb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:51.481400 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1442/2896] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:38:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:38:51.491283 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e730d00-225f-41fe-84b1-01bdecdd5152 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:51.492697 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6e730d00-225f-41fe-84b1-01bdecdd5152 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:51.548689 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e730d00-225f-41fe-84b1-01bdecdd5152 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:51.552341 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f9218c-e563-498e-beb1-ebbabf715a20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:51.552937 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f9218c-e563-498e-beb1-ebbabf715a20 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:51.553217 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f9218c-e563-498e-beb1-ebbabf715a20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:51.553517 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f9218c-e563-498e-beb1-ebbabf715a20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:51.562226 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f9218c-e563-498e-beb1-ebbabf715a20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:51.563646 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f9218c-e563-498e-beb1-ebbabf715a20 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:51.573633 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1455/2897] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:38:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:38:51.832618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c6905-51ed-4b08-8515-2580e701ccb7 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:51.834106 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1c6c6905-51ed-4b08-8515-2580e701ccb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:51.844580 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:38:51.851467 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c6905-51ed-4b08-8515-2580e701ccb7 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:51.857872 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb70ff37-5d2c-4024-bc0b-cb6f354dec5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:51.858935 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb70ff37-5d2c-4024-bc0b-cb6f354dec5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:51.859828 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb70ff37-5d2c-4024-bc0b-cb6f354dec5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:51.860112 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb70ff37-5d2c-4024-bc0b-cb6f354dec5e admin admin] PATH_INFO: `/v3/projects/4226079fb92c49e39d50cd567d438cae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:51.861376 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb70ff37-5d2c-4024-bc0b-cb6f354dec5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4226079fb92c49e39d50cd567d438cae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:51.863796 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb70ff37-5d2c-4024-bc0b-cb6f354dec5e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:51.910412 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-cb70ff37-5d2c-4024-bc0b-cb6f354dec5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4226079fb92c49e39d50cd567d438cae', 'request_id': 'req-cb70ff37-5d2c-4024-bc0b-cb6f354dec5e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:51.915869 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-cb70ff37-5d2c-4024-bc0b-cb6f354dec5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4226079fb92c49e39d50cd567d438cae', 'request_id': 'req-cb70ff37-5d2c-4024-bc0b-cb6f354dec5e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:51.922565 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-cb70ff37-5d2c-4024-bc0b-cb6f354dec5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4226079fb92c49e39d50cd567d438cae', 'request_id': 'req-cb70ff37-5d2c-4024-bc0b-cb6f354dec5e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:51.935393 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-cb70ff37-5d2c-4024-bc0b-cb6f354dec5e admin admin] The token cache is being invalidate because project 4226079fb92c49e39d50cd567d438cae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:38:51.935684 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-cb70ff37-5d2c-4024-bc0b-cb6f354dec5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:51.936307 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-cb70ff37-5d2c-4024-bc0b-cb6f354dec5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:38:51.938039 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1443/2898] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:38:51 2022] DELETE /identity/v3/projects/4226079fb92c49e39d50cd567d438cae => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:38:51.942751 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ffe3d-5724-4431-a71a-329a52e0e7d5 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:51.943033 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ffe3d-5724-4431-a71a-329a52e0e7d5 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:51.943305 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ffe3d-5724-4431-a71a-329a52e0e7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:51.962526 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-950ffe3d-5724-4431-a71a-329a52e0e7d5 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:38:51.963697 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-950ffe3d-5724-4431-a71a-329a52e0e7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:52.021744 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1456/2899] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:38:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:38:52.029692 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25674d-f5b7-435e-aaab-1d83ffcb4c47 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:52.029982 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25674d-f5b7-435e-aaab-1d83ffcb4c47 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:52.030287 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25674d-f5b7-435e-aaab-1d83ffcb4c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:52.051978 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-9f25674d-f5b7-435e-aaab-1d83ffcb4c47 None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:38:52.053211 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9f25674d-f5b7-435e-aaab-1d83ffcb4c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:52.061142 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1444/2900] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:38:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:38:52.065298 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ffe3d-5724-4431-a71a-329a52e0e7d5 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:52.075491 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ffe3d-5724-4431-a71a-329a52e0e7d5 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:52.080050 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a313e-2d78-49d1-aa22-ecb35b7ffa97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:52.080647 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a313e-2d78-49d1-aa22-ecb35b7ffa97 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:52.080928 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a313e-2d78-49d1-aa22-ecb35b7ffa97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:52.081202 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a313e-2d78-49d1-aa22-ecb35b7ffa97 admin admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:52.081883 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a313e-2d78-49d1-aa22-ecb35b7ffa97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:38:52.082209 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a313e-2d78-49d1-aa22-ecb35b7ffa97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:52.083836 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a313e-2d78-49d1-aa22-ecb35b7ffa97 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:52.095468 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1457/2901] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Sep 9 20:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:38:52.099816 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f25674d-f5b7-435e-aaab-1d83ffcb4c47 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:52.109033 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f25674d-f5b7-435e-aaab-1d83ffcb4c47 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:52.114561 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9829cc1c-3d0f-4f8f-a650-4844f94113f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:52.115158 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829cc1c-3d0f-4f8f-a650-4844f94113f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:52.115439 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829cc1c-3d0f-4f8f-a650-4844f94113f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:52.115714 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829cc1c-3d0f-4f8f-a650-4844f94113f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:52.116341 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9829cc1c-3d0f-4f8f-a650-4844f94113f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:52.118281 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9829cc1c-3d0f-4f8f-a650-4844f94113f9 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:52.144986 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1445/2902] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Sep 9 20:38:52 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:38:52.149049 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a313e-2d78-49d1-aa22-ecb35b7ffa97 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:52.158827 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a313e-2d78-49d1-aa22-ecb35b7ffa97 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:52.163534 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:38:52.227584 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02265e96-7b5c-409e-bf34-8eaee27978b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:52.228184 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02265e96-7b5c-409e-bf34-8eaee27978b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:52.228462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02265e96-7b5c-409e-bf34-8eaee27978b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:52.228732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02265e96-7b5c-409e-bf34-8eaee27978b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:52.229390 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02265e96-7b5c-409e-bf34-8eaee27978b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:52.231329 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02265e96-7b5c-409e-bf34-8eaee27978b9 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:52.252618 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1458/2903] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Sep 9 20:38:52 2022] POST /identity/v3/users => generated 291 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:38:52.257491 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9829cc1c-3d0f-4f8f-a650-4844f94113f9 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:52.268530 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9829cc1c-3d0f-4f8f-a650-4844f94113f9 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:52.273120 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4767282c-31f5-4a64-b8c3-d25961faf828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:52.273746 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4767282c-31f5-4a64-b8c3-d25961faf828 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:52.274028 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4767282c-31f5-4a64-b8c3-d25961faf828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:52.274301 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4767282c-31f5-4a64-b8c3-d25961faf828 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:52.274932 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4767282c-31f5-4a64-b8c3-d25961faf828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:52.276506 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4767282c-31f5-4a64-b8c3-d25961faf828 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:52.286529 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1446/2904] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:38:52.290211 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02265e96-7b5c-409e-bf34-8eaee27978b9 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:52.299908 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02265e96-7b5c-409e-bf34-8eaee27978b9 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:52.304979 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44584a0-ba40-4d7c-be02-c9fc88241739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:52.305659 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44584a0-ba40-4d7c-be02-c9fc88241739 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:52.305937 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44584a0-ba40-4d7c-be02-c9fc88241739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:52.306392 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44584a0-ba40-4d7c-be02-c9fc88241739 admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:52.307371 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44584a0-ba40-4d7c-be02-c9fc88241739 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:52.309102 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44584a0-ba40-4d7c-be02-c9fc88241739 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:52.320039 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1459/2905] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Sep 9 20:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:38:52.324102 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4767282c-31f5-4a64-b8c3-d25961faf828 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:52.334006 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4767282c-31f5-4a64-b8c3-d25961faf828 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:52.338556 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076cd83-4523-4253-90e5-bb58278835dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:52.339355 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076cd83-4523-4253-90e5-bb58278835dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:52.339638 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076cd83-4523-4253-90e5-bb58278835dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:52.339909 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076cd83-4523-4253-90e5-bb58278835dc admin admin] PATH_INFO: `/v3/projects/bbf97f61be434378b04b3f95da86e521/users/38988b326874436188346c19df131298/roles/63fc659c11144d369a3f3fa684e056ae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:52.354587 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1076cd83-4523-4253-90e5-bb58278835dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbf97f61be434378b04b3f95da86e521, user_id=38988b326874436188346c19df131298, role_id=63fc659c11144d369a3f3fa684e056ae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:52.357183 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1076cd83-4523-4253-90e5-bb58278835dc admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:52.370722 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1447/2906] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Sep 9 20:38:52 2022] PUT /identity/v3/projects/bbf97f61be434378b04b3f95da86e521/users/38988b326874436188346c19df131298/roles/63fc659c11144d369a3f3fa684e056ae => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:38:52.375627 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509e96d-2300-4258-b36a-ac45499ca00e None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:52.375907 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509e96d-2300-4258-b36a-ac45499ca00e None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:52.376176 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509e96d-2300-4258-b36a-ac45499ca00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:52.394369 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-8509e96d-2300-4258-b36a-ac45499ca00e None None] MFA Rules not processed for user `b20f75fb724246318736ac6f2901b55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:38:52.395546 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-8509e96d-2300-4258-b36a-ac45499ca00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:52.453697 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1460/2907] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:38:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:38:52.463779 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076cd83-4523-4253-90e5-bb58278835dc admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:52.465546 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1076cd83-4523-4253-90e5-bb58278835dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:52.528605 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076cd83-4523-4253-90e5-bb58278835dc admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:52.532221 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b79cf-025f-4d24-abed-ce0459f8e020 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:52.532826 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b79cf-025f-4d24-abed-ce0459f8e020 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:52.533104 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b79cf-025f-4d24-abed-ce0459f8e020 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:52.533404 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b79cf-025f-4d24-abed-ce0459f8e020 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:52.542268 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875b79cf-025f-4d24-abed-ce0459f8e020 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:52.543673 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875b79cf-025f-4d24-abed-ce0459f8e020 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:52.552861 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1448/2908] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:38:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:38:53.161371 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee4cb8-746c-4403-aaad-2d32a4df7bbf None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:53.161661 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee4cb8-746c-4403-aaad-2d32a4df7bbf None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:53.161932 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee4cb8-746c-4403-aaad-2d32a4df7bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:53.190340 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-1aee4cb8-746c-4403-aaad-2d32a4df7bbf None None] MFA Rules not processed for user `38988b326874436188346c19df131298`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:38:53.192088 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-1aee4cb8-746c-4403-aaad-2d32a4df7bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:53.258617 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1461/2909] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:38:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:38:53.270075 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b79cf-025f-4d24-abed-ce0459f8e020 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:38:53.271499 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-875b79cf-025f-4d24-abed-ce0459f8e020 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:53.299580 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:38:53.333512 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b79cf-025f-4d24-abed-ce0459f8e020 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:38:53.337098 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03550a8-19a1-4ed8-8573-b62e6a500a83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d86d2df1cb485592e8a5d7387131ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:38:53.337738 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03550a8-19a1-4ed8-8573-b62e6a500a83 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:53.338018 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03550a8-19a1-4ed8-8573-b62e6a500a83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:53.338291 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03550a8-19a1-4ed8-8573-b62e6a500a83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:53.346666 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03550a8-19a1-4ed8-8573-b62e6a500a83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:38:53.348112 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03550a8-19a1-4ed8-8573-b62e6a500a83 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:38:53.357113 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1449/2910] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:38:53 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:38:54.712820 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a39c63-a040-4160-8259-7f0f72478f36 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:38:54.713330 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a39c63-a040-4160-8259-7f0f72478f36 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:38:54.713432 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a39c63-a040-4160-8259-7f0f72478f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:38:54.733424 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-33a39c63-a040-4160-8259-7f0f72478f36 None None] MFA Rules not processed for user `38988b326874436188346c19df131298`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:38:54.734882 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-33a39c63-a040-4160-8259-7f0f72478f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:38:54.742252 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1462/2911] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:38:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:39:03.352885 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=15 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:39:03.608809 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03550a8-19a1-4ed8-8573-b62e6a500a83 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:39:03.610695 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e03550a8-19a1-4ed8-8573-b62e6a500a83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:39:03.692234 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03550a8-19a1-4ed8-8573-b62e6a500a83 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:39:03.697825 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d71d7b8-962f-4622-b2dd-05504e4f38a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:39:03.698426 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71d7b8-962f-4622-b2dd-05504e4f38a3 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:39:03.698705 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71d7b8-962f-4622-b2dd-05504e4f38a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:39:03.698972 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71d7b8-962f-4622-b2dd-05504e4f38a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:39:03.700679 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8d71d7b8-962f-4622-b2dd-05504e4f38a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:39:03.712635 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d71d7b8-962f-4622-b2dd-05504e4f38a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:39:03.714212 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d71d7b8-962f-4622-b2dd-05504e4f38a3 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:39:03.725035 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1450/2912] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:39:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:39:04.727047 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:39:08.208295 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a39c63-a040-4160-8259-7f0f72478f36 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:39:08.218045 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a39c63-a040-4160-8259-7f0f72478f36 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:39:08.222058 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d685b-de5b-4528-8936-f42a3b4ba7f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:39:08.222769 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d685b-de5b-4528-8936-f42a3b4ba7f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:39:08.223059 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d685b-de5b-4528-8936-f42a3b4ba7f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:39:08.223332 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d685b-de5b-4528-8936-f42a3b4ba7f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:39:08.225394 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-671d685b-de5b-4528-8936-f42a3b4ba7f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:39:08.235835 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671d685b-de5b-4528-8936-f42a3b4ba7f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:39:08.237279 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671d685b-de5b-4528-8936-f42a3b4ba7f5 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:39:08.245328 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1463/2913] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:39:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:39:08.364918 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d71d7b8-962f-4622-b2dd-05504e4f38a3 service nova] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:39:08.366429 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8d71d7b8-962f-4622-b2dd-05504e4f38a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:39:08.379119 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d71d7b8-962f-4622-b2dd-05504e4f38a3 service nova] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:39:08.383146 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb30ea0-0d7c-4cd0-b14a-8aa880ec4ef5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:39:08.383672 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb30ea0-0d7c-4cd0-b14a-8aa880ec4ef5 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:39:08.383951 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb30ea0-0d7c-4cd0-b14a-8aa880ec4ef5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:39:08.384223 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb30ea0-0d7c-4cd0-b14a-8aa880ec4ef5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:39:08.386032 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3fb30ea0-0d7c-4cd0-b14a-8aa880ec4ef5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:39:08.396828 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb30ea0-0d7c-4cd0-b14a-8aa880ec4ef5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:39:08.398362 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb30ea0-0d7c-4cd0-b14a-8aa880ec4ef5 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:39:08.409142 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1451/2914] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:39:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:39:08.498936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d685b-de5b-4528-8936-f42a3b4ba7f5 service neutron] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:39:08.507888 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d685b-de5b-4528-8936-f42a3b4ba7f5 service neutron] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:39:08.511773 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b75bd-adc7-4252-b2d2-0ed2fe039601 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:39:08.512364 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b75bd-adc7-4252-b2d2-0ed2fe039601 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:39:08.512640 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b75bd-adc7-4252-b2d2-0ed2fe039601 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:39:08.512908 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b75bd-adc7-4252-b2d2-0ed2fe039601 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:39:08.514916 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-f94b75bd-adc7-4252-b2d2-0ed2fe039601 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:39:08.529548 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b75bd-adc7-4252-b2d2-0ed2fe039601 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:39:08.530975 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b75bd-adc7-4252-b2d2-0ed2fe039601 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:39:08.542034 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1464/2915] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:39:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:39:18.404233 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=40 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:39:18.536817 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=12 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:39:57.983938 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41ea45-1795-4eb5-a3bc-7e4bf37884f5 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:39:57.984468 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41ea45-1795-4eb5-a3bc-7e4bf37884f5 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:39:57.984468 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41ea45-1795-4eb5-a3bc-7e4bf37884f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:39:58.004332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ba41ea45-1795-4eb5-a3bc-7e4bf37884f5 None None] MFA Rules not processed for user `38988b326874436188346c19df131298`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:39:58.005537 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ba41ea45-1795-4eb5-a3bc-7e4bf37884f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:39:58.012146 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1452/2916] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:39:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:40:07.998426 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:40:35.065942 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7378b-5061-438d-b3b5-a866a5c98a2a None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:40:35.066476 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7378b-5061-438d-b3b5-a866a5c98a2a None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:40:35.066581 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7378b-5061-438d-b3b5-a866a5c98a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:40:35.085020 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-28b7378b-5061-438d-b3b5-a866a5c98a2a None None] MFA Rules not processed for user `63371d392c8c4b73939ee73a1627c8dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:40:35.086244 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-28b7378b-5061-438d-b3b5-a866a5c98a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:40:35.140731 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1465/2917] 10.222.0.26 () {58 vars in 958 bytes} [Fri Sep 9 20:40:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:40:38.063992 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba41ea45-1795-4eb5-a3bc-7e4bf37884f5 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:40:38.065440 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ba41ea45-1795-4eb5-a3bc-7e4bf37884f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:40:38.081479 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba41ea45-1795-4eb5-a3bc-7e4bf37884f5 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:40:38.087440 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c50170-4b87-49ee-bdc0-71ba5581a1c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:40:38.088032 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c50170-4b87-49ee-bdc0-71ba5581a1c4 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:40:38.088312 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c50170-4b87-49ee-bdc0-71ba5581a1c4 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:40:38.088607 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c50170-4b87-49ee-bdc0-71ba5581a1c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:40:38.090284 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-66c50170-4b87-49ee-bdc0-71ba5581a1c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:40:38.100022 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c50170-4b87-49ee-bdc0-71ba5581a1c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:40:38.101612 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c50170-4b87-49ee-bdc0-71ba5581a1c4 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:40:38.110162 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1453/2918] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:40:38 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:40:45.134220 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=25 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:40:48.106416 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=6 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:40:54.611190 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b7378b-5061-438d-b3b5-a866a5c98a2a None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:40:54.612836 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-28b7378b-5061-438d-b3b5-a866a5c98a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:40:54.714344 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b7378b-5061-438d-b3b5-a866a5c98a2a None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:40:54.719499 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19624b4d-74bb-4dc5-b94c-5cb3910ca459 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26d24cf0a21d491082b0c2f11e749446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:40:54.720108 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19624b4d-74bb-4dc5-b94c-5cb3910ca459 service placement] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:40:54.720390 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19624b4d-74bb-4dc5-b94c-5cb3910ca459 service placement] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:40:54.720666 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19624b4d-74bb-4dc5-b94c-5cb3910ca459 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:40:54.722453 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-19624b4d-74bb-4dc5-b94c-5cb3910ca459 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:40:54.734201 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19624b4d-74bb-4dc5-b94c-5cb3910ca459 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:40:54.735796 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19624b4d-74bb-4dc5-b94c-5cb3910ca459 service placement] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:40:54.747630 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1466/2919] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:40:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:40:56.837675 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c50170-4b87-49ee-bdc0-71ba5581a1c4 service nova] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:40:56.851147 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c50170-4b87-49ee-bdc0-71ba5581a1c4 service nova] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:40:56.858196 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d162eb7-f7d7-4ffd-8f03-d464ebafa70d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ece6128f9245f7b49f3e7a9faaed22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:40:56.858988 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d162eb7-f7d7-4ffd-8f03-d464ebafa70d service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:40:56.859436 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d162eb7-f7d7-4ffd-8f03-d464ebafa70d service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:40:56.859876 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d162eb7-f7d7-4ffd-8f03-d464ebafa70d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:40:56.862098 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5d162eb7-f7d7-4ffd-8f03-d464ebafa70d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:40:56.875332 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d162eb7-f7d7-4ffd-8f03-d464ebafa70d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:40:56.877227 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d162eb7-f7d7-4ffd-8f03-d464ebafa70d service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:40:56.889635 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1454/2920] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:40:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:40:58.293291 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19624b4d-74bb-4dc5-b94c-5cb3910ca459 service placement] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:40:58.306391 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19624b4d-74bb-4dc5-b94c-5cb3910ca459 service placement] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:40:58.312508 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c0b77-5b20-4066-b4b8-4e5618c5669f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:40:58.313499 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c0b77-5b20-4066-b4b8-4e5618c5669f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:40:58.313793 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c0b77-5b20-4066-b4b8-4e5618c5669f admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:40:58.314053 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c0b77-5b20-4066-b4b8-4e5618c5669f admin admin] PATH_INFO: `/v3/users/38988b326874436188346c19df131298` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:40:58.315216 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c0b77-5b20-4066-b4b8-4e5618c5669f admin admin] RBAC: Authorizing `identity:delete_user(user_id=38988b326874436188346c19df131298)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:40:58.317758 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c0b77-5b20-4066-b4b8-4e5618c5669f admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:40:58.393838 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-d01c0b77-5b20-4066-b4b8-4e5618c5669f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38988b326874436188346c19df131298'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:40:58.411005 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-d01c0b77-5b20-4066-b4b8-4e5618c5669f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38988b326874436188346c19df131298'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:40:58.420040 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-d01c0b77-5b20-4066-b4b8-4e5618c5669f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38988b326874436188346c19df131298'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:40:58.431311 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-d01c0b77-5b20-4066-b4b8-4e5618c5669f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38988b326874436188346c19df131298'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:40:58.431949 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-d01c0b77-5b20-4066-b4b8-4e5618c5669f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38988b326874436188346c19df131298'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:40:58.439141 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1467/2921] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:40:58 2022] DELETE /identity/v3/users/38988b326874436188346c19df131298 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:40:58.603139 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d162eb7-f7d7-4ffd-8f03-d464ebafa70d service nova] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:40:58.604780 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5d162eb7-f7d7-4ffd-8f03-d464ebafa70d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:40:58.622778 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d162eb7-f7d7-4ffd-8f03-d464ebafa70d service nova] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:40:58.624833 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c0b77-5b20-4066-b4b8-4e5618c5669f admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:40:58.626350 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-d01c0b77-5b20-4066-b4b8-4e5618c5669f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:40:58.690843 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12dae6-a52f-4846-9ade-c52e31f09dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:40:58.691572 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12dae6-a52f-4846-9ade-c52e31f09dda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:40:58.691849 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12dae6-a52f-4846-9ade-c52e31f09dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:40:58.692121 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12dae6-a52f-4846-9ade-c52e31f09dda admin admin] PATH_INFO: `/v3/users/3540adbd8efb4a7a97ca44a6c0e5e51d` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:40:58.693297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12dae6-a52f-4846-9ade-c52e31f09dda admin admin] RBAC: Authorizing `identity:delete_user(user_id=3540adbd8efb4a7a97ca44a6c0e5e51d)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:40:58.695195 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12dae6-a52f-4846-9ade-c52e31f09dda admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:40:58.697955 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c0b77-5b20-4066-b4b8-4e5618c5669f admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:40:58.702478 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c8dfb-e2d3-4215-9a11-721b7ba895d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:40:58.703109 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c8dfb-e2d3-4215-9a11-721b7ba895d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:40:58.703389 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c8dfb-e2d3-4215-9a11-721b7ba895d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:40:58.703661 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c8dfb-e2d3-4215-9a11-721b7ba895d2 admin admin] PATH_INFO: `/v3/projects/bbf97f61be434378b04b3f95da86e521` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:40:58.704765 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8c8dfb-e2d3-4215-9a11-721b7ba895d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbf97f61be434378b04b3f95da86e521)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:40:58.707630 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8c8dfb-e2d3-4215-9a11-721b7ba895d2 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:40:58.709688 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:40:58.745980 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-cc12dae6-a52f-4846-9ade-c52e31f09dda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3540adbd8efb4a7a97ca44a6c0e5e51d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:40:58.753132 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-cc12dae6-a52f-4846-9ade-c52e31f09dda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3540adbd8efb4a7a97ca44a6c0e5e51d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:40:58.755558 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-ec8c8dfb-e2d3-4215-9a11-721b7ba895d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbf97f61be434378b04b3f95da86e521', 'request_id': 'req-ec8c8dfb-e2d3-4215-9a11-721b7ba895d2'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:40:58.763549 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-ec8c8dfb-e2d3-4215-9a11-721b7ba895d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbf97f61be434378b04b3f95da86e521', 'request_id': 'req-ec8c8dfb-e2d3-4215-9a11-721b7ba895d2'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:40:58.765432 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-cc12dae6-a52f-4846-9ade-c52e31f09dda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3540adbd8efb4a7a97ca44a6c0e5e51d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:40:58.769268 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-ec8c8dfb-e2d3-4215-9a11-721b7ba895d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbf97f61be434378b04b3f95da86e521', 'request_id': 'req-ec8c8dfb-e2d3-4215-9a11-721b7ba895d2'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:40:58.773942 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-cc12dae6-a52f-4846-9ade-c52e31f09dda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3540adbd8efb4a7a97ca44a6c0e5e51d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:40:58.774556 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-cc12dae6-a52f-4846-9ade-c52e31f09dda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3540adbd8efb4a7a97ca44a6c0e5e51d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:40:58.780526 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1455/2922] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:40:58 2022] DELETE /identity/v3/users/3540adbd8efb4a7a97ca44a6c0e5e51d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:40:58.784941 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-ec8c8dfb-e2d3-4215-9a11-721b7ba895d2 admin admin] The token cache is being invalidate because project bbf97f61be434378b04b3f95da86e521 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:40:58.785815 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-ec8c8dfb-e2d3-4215-9a11-721b7ba895d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:40:58.786462 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-ec8c8dfb-e2d3-4215-9a11-721b7ba895d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:40:58.787932 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1468/2923] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:40:58 2022] DELETE /identity/v3/projects/bbf97f61be434378b04b3f95da86e521 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:40:58.979604 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12dae6-a52f-4846-9ade-c52e31f09dda admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:40:58.981154 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-cc12dae6-a52f-4846-9ade-c52e31f09dda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:40:59.056516 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12dae6-a52f-4846-9ade-c52e31f09dda admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:40:59.061115 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edffe5f-25dd-4404-b45a-207502840736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:40:59.061777 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edffe5f-25dd-4404-b45a-207502840736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:40:59.062057 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edffe5f-25dd-4404-b45a-207502840736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:40:59.062329 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edffe5f-25dd-4404-b45a-207502840736 admin admin] PATH_INFO: `/v3/projects/474aa97f67a8457ab6900081711a0bf1` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:40:59.063492 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edffe5f-25dd-4404-b45a-207502840736 admin admin] RBAC: Authorizing `identity:delete_project(project_id=474aa97f67a8457ab6900081711a0bf1)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:40:59.065410 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edffe5f-25dd-4404-b45a-207502840736 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:40:59.073887 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:40:59.114619 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7edffe5f-25dd-4404-b45a-207502840736 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '474aa97f67a8457ab6900081711a0bf1', 'request_id': 'req-7edffe5f-25dd-4404-b45a-207502840736'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:40:59.120401 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7edffe5f-25dd-4404-b45a-207502840736 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '474aa97f67a8457ab6900081711a0bf1', 'request_id': 'req-7edffe5f-25dd-4404-b45a-207502840736'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:40:59.126711 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7edffe5f-25dd-4404-b45a-207502840736 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '474aa97f67a8457ab6900081711a0bf1', 'request_id': 'req-7edffe5f-25dd-4404-b45a-207502840736'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:40:59.140216 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7edffe5f-25dd-4404-b45a-207502840736 admin admin] The token cache is being invalidate because project 474aa97f67a8457ab6900081711a0bf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:40:59.140732 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7edffe5f-25dd-4404-b45a-207502840736 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:40:59.141221 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7edffe5f-25dd-4404-b45a-207502840736 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:40:59.142827 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1456/2924] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:40:58 2022] DELETE /identity/v3/projects/474aa97f67a8457ab6900081711a0bf1 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:40:59.147455 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c8dfb-e2d3-4215-9a11-721b7ba895d2 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:40:59.148880 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-ec8c8dfb-e2d3-4215-9a11-721b7ba895d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:40:59.211289 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c8dfb-e2d3-4215-9a11-721b7ba895d2 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:40:59.268116 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb654ce-d7f1-43ce-ba22-e984e8d31537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:40:59.268750 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb654ce-d7f1-43ce-ba22-e984e8d31537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:40:59.269027 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb654ce-d7f1-43ce-ba22-e984e8d31537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:40:59.269324 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb654ce-d7f1-43ce-ba22-e984e8d31537 admin admin] PATH_INFO: `/v3/users/df09a37fed3140d2a07663e2425dc463` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:40:59.271202 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb654ce-d7f1-43ce-ba22-e984e8d31537 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df09a37fed3140d2a07663e2425dc463)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:40:59.273115 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb654ce-d7f1-43ce-ba22-e984e8d31537 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:40:59.315374 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:40:59.322837 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-2cb654ce-d7f1-43ce-ba22-e984e8d31537 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df09a37fed3140d2a07663e2425dc463'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:40:59.338476 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-2cb654ce-d7f1-43ce-ba22-e984e8d31537 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df09a37fed3140d2a07663e2425dc463'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:40:59.345597 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-2cb654ce-d7f1-43ce-ba22-e984e8d31537 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df09a37fed3140d2a07663e2425dc463'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:40:59.353062 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-2cb654ce-d7f1-43ce-ba22-e984e8d31537 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df09a37fed3140d2a07663e2425dc463'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:40:59.353764 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-2cb654ce-d7f1-43ce-ba22-e984e8d31537 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df09a37fed3140d2a07663e2425dc463'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:40:59.359528 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1469/2925] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:40:59 2022] DELETE /identity/v3/users/df09a37fed3140d2a07663e2425dc463 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:40:59.529725 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edffe5f-25dd-4404-b45a-207502840736 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:40:59.531709 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7edffe5f-25dd-4404-b45a-207502840736 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:40:59.546462 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edffe5f-25dd-4404-b45a-207502840736 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:40:59.601902 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d2c3b-2331-4e18-9f32-4e38470f4b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:40:59.602531 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d2c3b-2331-4e18-9f32-4e38470f4b77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:40:59.602805 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d2c3b-2331-4e18-9f32-4e38470f4b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:40:59.603073 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d2c3b-2331-4e18-9f32-4e38470f4b77 admin admin] PATH_INFO: `/v3/projects/cdecc6100c5b4f7385eb7b83331aafbe` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:40:59.604169 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d2c3b-2331-4e18-9f32-4e38470f4b77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdecc6100c5b4f7385eb7b83331aafbe)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:40:59.606111 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d2c3b-2331-4e18-9f32-4e38470f4b77 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:40:59.652907 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-147d2c3b-2331-4e18-9f32-4e38470f4b77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdecc6100c5b4f7385eb7b83331aafbe', 'request_id': 'req-147d2c3b-2331-4e18-9f32-4e38470f4b77'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:40:59.658530 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-147d2c3b-2331-4e18-9f32-4e38470f4b77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdecc6100c5b4f7385eb7b83331aafbe', 'request_id': 'req-147d2c3b-2331-4e18-9f32-4e38470f4b77'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:40:59.664914 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-147d2c3b-2331-4e18-9f32-4e38470f4b77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdecc6100c5b4f7385eb7b83331aafbe', 'request_id': 'req-147d2c3b-2331-4e18-9f32-4e38470f4b77'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:40:59.679089 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-147d2c3b-2331-4e18-9f32-4e38470f4b77 admin admin] The token cache is being invalidate because project cdecc6100c5b4f7385eb7b83331aafbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:40:59.679380 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-147d2c3b-2331-4e18-9f32-4e38470f4b77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:40:59.679996 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-147d2c3b-2331-4e18-9f32-4e38470f4b77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:40:59.681510 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1457/2926] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:40:59 2022] DELETE /identity/v3/projects/cdecc6100c5b4f7385eb7b83331aafbe => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:41:09.356123 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=14,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:41:09.675306 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=72,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:41:15.508947 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb654ce-d7f1-43ce-ba22-e984e8d31537 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:41:15.510379 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-2cb654ce-d7f1-43ce-ba22-e984e8d31537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:41:15.586824 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb654ce-d7f1-43ce-ba22-e984e8d31537 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:41:15.647526 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a3889e-a5c9-4a6f-8c18-c5669bf4769c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26d24cf0a21d491082b0c2f11e749446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:41:15.648135 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3889e-a5c9-4a6f-8c18-c5669bf4769c service placement] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:41:15.648420 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3889e-a5c9-4a6f-8c18-c5669bf4769c service placement] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:41:15.648695 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3889e-a5c9-4a6f-8c18-c5669bf4769c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:41:15.650591 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-18a3889e-a5c9-4a6f-8c18-c5669bf4769c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:41:15.662865 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a3889e-a5c9-4a6f-8c18-c5669bf4769c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:41:15.664493 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a3889e-a5c9-4a6f-8c18-c5669bf4769c service placement] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:41:15.676173 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1470/2927] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:41:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:41:25.671115 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=58 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:42:37.549552 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d2c3b-2331-4e18-9f32-4e38470f4b77 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:42:37.551003 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-147d2c3b-2331-4e18-9f32-4e38470f4b77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:42:37.607086 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d2c3b-2331-4e18-9f32-4e38470f4b77 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:42:37.610658 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957f1cd-8508-46f1-b3f6-629aba10f0a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c6e51f2694bf5ae7ea097cf33fb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:42:37.611254 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957f1cd-8508-46f1-b3f6-629aba10f0a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:42:37.611557 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957f1cd-8508-46f1-b3f6-629aba10f0a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:42:37.611839 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957f1cd-8508-46f1-b3f6-629aba10f0a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:42:37.614382 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a957f1cd-8508-46f1-b3f6-629aba10f0a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:42:37.677505 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a957f1cd-8508-46f1-b3f6-629aba10f0a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:42:37.678937 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a957f1cd-8508-46f1-b3f6-629aba10f0a1 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:42:37.688462 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1458/2928] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:42:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:42:40.673922 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a3889e-a5c9-4a6f-8c18-c5669bf4769c service placement] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:42:40.675430 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-18a3889e-a5c9-4a6f-8c18-c5669bf4769c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:42:40.688608 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a3889e-a5c9-4a6f-8c18-c5669bf4769c service placement] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:42:40.694558 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41577d-24fe-4cab-b1f3-9d73cb44c08d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26d24cf0a21d491082b0c2f11e749446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:42:40.695169 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41577d-24fe-4cab-b1f3-9d73cb44c08d service placement] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:42:40.695445 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41577d-24fe-4cab-b1f3-9d73cb44c08d service placement] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:42:40.695714 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41577d-24fe-4cab-b1f3-9d73cb44c08d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:42:40.697956 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6e41577d-24fe-4cab-b1f3-9d73cb44c08d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:42:40.710435 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e41577d-24fe-4cab-b1f3-9d73cb44c08d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:42:40.712561 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e41577d-24fe-4cab-b1f3-9d73cb44c08d service placement] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:42:40.727564 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1471/2929] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:42:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:42:43.579040 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957f1cd-8508-46f1-b3f6-629aba10f0a1 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:42:43.580519 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a957f1cd-8508-46f1-b3f6-629aba10f0a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:42:43.593740 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957f1cd-8508-46f1-b3f6-629aba10f0a1 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:42:43.598650 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38687af9-bac1-4ab9-9252-436fb53c61ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:42:43.599913 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38687af9-bac1-4ab9-9252-436fb53c61ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:42:43.600191 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38687af9-bac1-4ab9-9252-436fb53c61ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:42:43.600463 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38687af9-bac1-4ab9-9252-436fb53c61ab admin admin] PATH_INFO: `/v3/users/63371d392c8c4b73939ee73a1627c8dd` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:42:43.601648 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38687af9-bac1-4ab9-9252-436fb53c61ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=63371d392c8c4b73939ee73a1627c8dd)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:42:43.603561 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38687af9-bac1-4ab9-9252-436fb53c61ab admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:42:43.661238 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-38687af9-bac1-4ab9-9252-436fb53c61ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63371d392c8c4b73939ee73a1627c8dd'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:42:43.668291 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-38687af9-bac1-4ab9-9252-436fb53c61ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63371d392c8c4b73939ee73a1627c8dd'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:42:43.680371 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-38687af9-bac1-4ab9-9252-436fb53c61ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63371d392c8c4b73939ee73a1627c8dd'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:42:43.687825 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-38687af9-bac1-4ab9-9252-436fb53c61ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63371d392c8c4b73939ee73a1627c8dd'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:42:43.688524 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-38687af9-bac1-4ab9-9252-436fb53c61ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63371d392c8c4b73939ee73a1627c8dd'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:42:43.694922 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1459/2930] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:42:43 2022] DELETE /identity/v3/users/63371d392c8c4b73939ee73a1627c8dd => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:42:43.862937 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41577d-24fe-4cab-b1f3-9d73cb44c08d service placement] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:42:43.873985 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41577d-24fe-4cab-b1f3-9d73cb44c08d service placement] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:42:43.931103 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554f23eb-f610-46d5-8be6-a8031ffdfa8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:42:43.931732 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f23eb-f610-46d5-8be6-a8031ffdfa8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:42:43.932054 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f23eb-f610-46d5-8be6-a8031ffdfa8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:42:43.932361 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f23eb-f610-46d5-8be6-a8031ffdfa8c admin admin] PATH_INFO: `/v3/projects/7e12f0c37bd14176b04dfd87d7b10520` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:42:43.933766 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554f23eb-f610-46d5-8be6-a8031ffdfa8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e12f0c37bd14176b04dfd87d7b10520)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:42:43.935707 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554f23eb-f610-46d5-8be6-a8031ffdfa8c admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:42:43.980414 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-554f23eb-f610-46d5-8be6-a8031ffdfa8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e12f0c37bd14176b04dfd87d7b10520', 'request_id': 'req-554f23eb-f610-46d5-8be6-a8031ffdfa8c'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:42:43.987189 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-554f23eb-f610-46d5-8be6-a8031ffdfa8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e12f0c37bd14176b04dfd87d7b10520', 'request_id': 'req-554f23eb-f610-46d5-8be6-a8031ffdfa8c'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:42:43.993365 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-554f23eb-f610-46d5-8be6-a8031ffdfa8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e12f0c37bd14176b04dfd87d7b10520', 'request_id': 'req-554f23eb-f610-46d5-8be6-a8031ffdfa8c'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:42:44.004131 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-554f23eb-f610-46d5-8be6-a8031ffdfa8c admin admin] The token cache is being invalidate because project 7e12f0c37bd14176b04dfd87d7b10520 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:42:44.004432 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-554f23eb-f610-46d5-8be6-a8031ffdfa8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:42:44.005044 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-554f23eb-f610-46d5-8be6-a8031ffdfa8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:42:44.007125 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1472/2931] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:42:43 2022] DELETE /identity/v3/projects/7e12f0c37bd14176b04dfd87d7b10520 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:42:44.011310 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38687af9-bac1-4ab9-9252-436fb53c61ab admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:42:44.012750 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-38687af9-bac1-4ab9-9252-436fb53c61ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:42:44.025618 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38687af9-bac1-4ab9-9252-436fb53c61ab admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:42:44.043651 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:42:44.083208 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7d5b0-6643-4825-a7a6-1b2e4c089059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:42:44.083895 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7d5b0-6643-4825-a7a6-1b2e4c089059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:42:44.084203 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7d5b0-6643-4825-a7a6-1b2e4c089059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:42:44.084482 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7d5b0-6643-4825-a7a6-1b2e4c089059 admin admin] PATH_INFO: `/v3/users/7114fe9707834ac1a3b03b4b41c3b36a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:42:44.085665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7d5b0-6643-4825-a7a6-1b2e4c089059 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7114fe9707834ac1a3b03b4b41c3b36a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:42:44.087579 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7d5b0-6643-4825-a7a6-1b2e4c089059 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:42:44.133297 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-72d7d5b0-6643-4825-a7a6-1b2e4c089059 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7114fe9707834ac1a3b03b4b41c3b36a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:42:44.140125 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-72d7d5b0-6643-4825-a7a6-1b2e4c089059 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7114fe9707834ac1a3b03b4b41c3b36a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:42:44.151942 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-72d7d5b0-6643-4825-a7a6-1b2e4c089059 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7114fe9707834ac1a3b03b4b41c3b36a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:42:44.158322 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-72d7d5b0-6643-4825-a7a6-1b2e4c089059 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7114fe9707834ac1a3b03b4b41c3b36a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:42:44.158987 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-72d7d5b0-6643-4825-a7a6-1b2e4c089059 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7114fe9707834ac1a3b03b4b41c3b36a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:42:44.164834 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1460/2932] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Sep 9 20:42:44 2022] DELETE /identity/v3/users/7114fe9707834ac1a3b03b4b41c3b36a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:42:44.336029 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554f23eb-f610-46d5-8be6-a8031ffdfa8c admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:42:44.337490 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-554f23eb-f610-46d5-8be6-a8031ffdfa8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:42:44.351234 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554f23eb-f610-46d5-8be6-a8031ffdfa8c admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:42:44.407037 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737452fc-52c0-48b1-9502-80ce8057b2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20f75fb724246318736ac6f2901b55a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8a4d19e4885495c84cf29b80391fd52', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:42:44.407662 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737452fc-52c0-48b1-9502-80ce8057b2be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:42:44.407936 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737452fc-52c0-48b1-9502-80ce8057b2be admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:42:44.408204 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737452fc-52c0-48b1-9502-80ce8057b2be admin admin] PATH_INFO: `/v3/projects/d8e762a2344248dc9e79ff464c26dddc` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:42:44.409311 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737452fc-52c0-48b1-9502-80ce8057b2be admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8e762a2344248dc9e79ff464c26dddc)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:42:44.411192 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737452fc-52c0-48b1-9502-80ce8057b2be admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:42:44.432557 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:42:44.457109 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-737452fc-52c0-48b1-9502-80ce8057b2be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8e762a2344248dc9e79ff464c26dddc', 'request_id': 'req-737452fc-52c0-48b1-9502-80ce8057b2be'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:42:44.464326 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-737452fc-52c0-48b1-9502-80ce8057b2be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8e762a2344248dc9e79ff464c26dddc', 'request_id': 'req-737452fc-52c0-48b1-9502-80ce8057b2be'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:42:44.470031 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-737452fc-52c0-48b1-9502-80ce8057b2be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8e762a2344248dc9e79ff464c26dddc', 'request_id': 'req-737452fc-52c0-48b1-9502-80ce8057b2be'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:42:44.481389 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-737452fc-52c0-48b1-9502-80ce8057b2be admin admin] The token cache is being invalidate because project d8e762a2344248dc9e79ff464c26dddc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81430) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 20:42:44.481680 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-737452fc-52c0-48b1-9502-80ce8057b2be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:42:44.482304 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-737452fc-52c0-48b1-9502-80ce8057b2be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 20:42:44.483763 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1473/2933] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Sep 9 20:42:44 2022] DELETE /identity/v3/projects/d8e762a2344248dc9e79ff464c26dddc => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:42:54.161331 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=43,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:42:54.478449 dv-cinder-849325-4-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:22.505060 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7d5b0-6643-4825-a7a6-1b2e4c089059 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:22.507087 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-72d7d5b0-6643-4825-a7a6-1b2e4c089059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:22.653667 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7d5b0-6643-4825-a7a6-1b2e4c089059 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:22.711925 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d9ffa9-0b93-4e87-894c-b64d2d0f533a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26d24cf0a21d491082b0c2f11e749446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '933820e2adbc473bbd57a4d59be75d72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:22.712589 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9ffa9-0b93-4e87-894c-b64d2d0f533a service placement] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:22.712867 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9ffa9-0b93-4e87-894c-b64d2d0f533a service placement] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:22.713144 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9ffa9-0b93-4e87-894c-b64d2d0f533a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:22.714982 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-39d9ffa9-0b93-4e87-894c-b64d2d0f533a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:22.726665 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d9ffa9-0b93-4e87-894c-b64d2d0f533a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:22.728173 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d9ffa9-0b93-4e87-894c-b64d2d0f533a service placement] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:22.737901 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1461/2934] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Sep 9 20:46:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:46:32.733675 dv-cinder-849325-4-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}